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Abstract 


There is a growing view that students who enroll in developmental courses are less 
successful in completing their programs than non-developmental students. Nevertheless, even 
though developmental students as a group ultimately might not be as academically successful as 
non-developmental students, many of them might still derive benefit from taking developmental 
courses. In this paper we address the question, “Does taking developmental courses benefit 
students at all, in the sense that they are more successful than they would have been if they had 
not taken developmental courses?” 

Data for the study consisted of ACT® Test and college outcomes data for over 118,000 
students who first enrolled in one of 75 two-year and four-year postsecondary institutions. We 
compared the success of students who initially enrolled in six developmental courses in English, 
mathematics, or reading with those of students who initially enrolled in associated higher-level 
courses. We first estimated probabilities of success with respect to twelve outcome variables 
ranging from performance in the associated higher-level college course to Bachelor’s degree 
completion in six years. The probabilities of success were conditioned on ACT Test score, 
enrollment status (full- or part-time), college type (two-year vs. four-year), and the grade 
received in the developmental course (if taken). We then compared the probabilities of success 
of students who did and did not take the developmental course, but who otherwise were similar. 

Like others, we found that the developmental students in this study were less successful 
as a group than the non-developmental students with respect to GP A/persistence over time and 
degree completion within a fixed time period. Further consideration of time to degree, however, 
showed that developmental students typically completed a Bachelor’s degree in six years at a 
rate similar to or higher than that of non-developmental students in five years. 



Particular subgroups of developmental students, as characterized by their ACT Test 
scores, the grade they received in the developmental course, and their enrollment status benefited 
from taking the developmental course. In particular, students who received an A (or sometimes 
a B) grade in the developmental course appeared to benefit from taking it. Moreover, part-time 
students appeared to derive more benefit from taking developmental courses than full-time 
students did. 

The report concludes with a discussion about the practical implications of these findings 
and possible contributing factors to academic success, such as the quality of developmental 
instruction, the time needed to complete a degree, and the noncognitive characteristics of 
developmental students. 


iii 



Acknowledgement 


The authors would like to thank Justine Radunzel for her assistance in preparing the data for this 
study and Jeff Allen, Justine Radunzel, and Jim Sconing for their comments and suggestions. 


IV 



V 



A Study of the Effectiveness of Developmental Courses 
for Improving Success in College 

Policy makers have in recent years increased their scrutiny of developmental instruction 
in college. Underlying their scrutiny is a long-held frustration that despite massive expenditures 
in K-12 education, students are graduating from high school unprepared to do college-level work 
(e.g., ACT, 2012a; Greene & Winters, 2005; “Student readiness: The challenge for colleges,” 
2006). As a result, the number of students who take developmental courses remains significantly 
high; about 36% of U.S. freshmen enroll in at least one developmental course upon entry to 
college (Adelman, 2004; Attewell, Lavin, Domina, & Levey, 2006; National Center for 
Education Statistics (NCES), 2008). Vandal (2010) reported that many states have remediation 
rates between 30 and 40%, and rates for some states exceed 50%. In the academic year 2009- 
2010, developmental instruction was provided by nearly all public two-year colleges, 75% of 
public four-year institutions, and 66% of private four-year institutions (NCES, 2010). 

Compounding policy makers’ frustration is the growing view that students who enroll in 
developmental courses are less successful than non-developmental students in completing their 
programs. For example, NCES (2004) found that 30% of 1992 12 th graders who enrolled in 
developmental coursework in college had completed a degree or certificate by 2000, compared to 
69% of non-developmental college enrollees. National Education Longitudinal Study (NELS) 
data also showed that less than 25% of community college students who enrolled in 
developmental education completed a degree or certification program within eight years of 
enrollment, compared to 40% of similar students who did not enroll in developmental education 
(Attewell et al., 2006). 
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Other studies, however, report more positive outcomes, with developmental students 
having a greater likelihood of completing a Bachelor’s degree than similar students who did not 
take developmental coursework (Adelman, 2006; Bettinger & Long, 2005a; Boylan, Bonham, & 
Bliss, 1992). Bettinger and Long (2005a) found that students who enrolled in developmental 
mathematics were 10% more likely to complete a Bachelor’s degree than students not enrolled in 
developmental mathematics. The gap was even wider for English, with a difference in likelihood 
of 17% favoring English developmental students over non-developmental students. Still other 
studies concluded that developmental instruction is beneficial for persistence to the second year 
(Calcagno and Long (2008) report a 2.0 to 3.8 percentage point difference), but not for later 
college outcomes (e.g., Boatman & Long, 2010). 

It is likely that the differences in the findings of these studies result from differences in 
their research design. 1 Important design characteristics are: 

• the stated or implied definition of developmental education (developmental coursework) 
vs. the combination of developmental coursework, support programs and services; 

• the college outcomes examined; 

• the characteristics of the students studied (e.g., high vs. low achievement, full-time 
students only, purpose for enrolling, etc.); 

• the methodology used (descriptive vs. regression modeling); or 

• the lack or inclusion of statistical controls for relevant group differences, including prior 
academic preparation. 

Most studies also do not consider that many students enroll in, but do not complete, 

developmental courses in the first place (Bailey, 2009; Russell, 2008; Sawyer & Schiel, 2000). 

1 See Adelman, 2006; Attewell et al., 2006; Bahr, 2008; Bettinger & Long, 2004; 2005a; Lewis & Farris, 1996; 
Schoenecker, Bollman, & Evens, 1996; and Weissman, Silk, & Bulakowski, 1995 for specific design details and 
comparisons of designs across studies. 
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Students are also frustrated. Not only are most developmental students ultimately 
unsuccessful; they incur substantial debt, loss of time and money, and student loan “hangovers” 
(Bailey, Jeong, & Cho, 2010, “Experts: Remedial classes need fixing,” 2012), as well as lower 
self-esteem, greater frustration, and higher dropout rates (Bettinger & Long, 2007). They may 
also reduce their eligibility for financial aid (Bettinger & Long, 2005b). 

further, the costs of postsecondary education have continued to increase while the ability 
of state and federal governments to subsidize it has eroded (Bettinger & Long, 2007; Merisotis & 
Phipps, 2000; Russell, 2008). Current annual costs to states and students for developmental 
instruction are estimated at $1.9 to $2.3 billion dollars at community colleges and $500 million 
at four-year colleges (Bailey et al., 2010; Strong American Schools, 2008). Individual state 
estimates fall in the tens of millions of dollars (Saxon & Boylan, 2001; Llorida Office of 
Program Policy Analysis and Government Accountability, 2006; Ohio Board of Regents, 2006). 

Policy makers are questioning whether in a time of constrained financial resources, 
subsidizing developmental education is a wise use of public money. Some states and college 
systems have already restricted developmental coursework to two-year colleges (e.g., Florida, 
Kansas, Illinois, the CUNY system). Others have placed limitations on developmental courses 
by four-year colleges (Bettinger & Long, 2005a; “Experts: Remedial classes need fixing,” 2012; 
Merisotis & Phipps, 2000; Saxon & Boylan, 2001). Parker (2007) reported that 22 states and 
systems had reduced or eliminated remedial coursework. 

Key considerations in discussions about developmental education are its costs and 
benefits. In this report, we consider questions related to benefits: To what extent do 
developmental courses adequately prepare students for standard first-year college courses? To 
what extent do they adequately prepare students for longer-term success in college (as measured 
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by retention, grade point average (GPA), degree completion, etc.)? The overarching questions 
are “Does developmental instruction benefit students, and how can we measure benefit?” 

Phipps (1998) identified four questions that postsecondary institutions need to answer in 
evaluating the effectiveness of developmental instruction: 

1 . Do students successfully complete developmental courses? 

2. Do students move from developmental instruction to college-level work? 

3. Do students who take developmental courses eventually complete college-level courses? 

4. Are developmental students persisting and reaching their academic goals? 

Implicit in these questions is the hope that students who successfully complete developmental 
courses will ultimately succeed in rates comparable to those of students who do not need to take 
developmental courses. 

Most research to date has compared the overall success of developmental students as a 
group with that of non-developmental students; a typical finding is that developmental students 
are not as successful in the long term as non-developmental students. We ask a different 
question: “Does taking developmental courses benefit students at all, in the sense that they are 
more successful than they would have been if they had not taken developmental courses?” This 
question relates to “value-added”: Even if taking developmental courses does not add enough 
value to make students as a group as likely to succeed as non-developmental students, does it add 
any value at all? Would students have been just as unlikely to succeed if they had not taken 
developmental courses? 

Aside from providing information about whether developmental instruction adds value to 
students’ attempts to succeed academically, answers to this question could focus attention on 
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determinants of success other than prior academic preparation and subsequent instruction. We 
describe other possible determinants in the discussion section of this report. 

Ideally, one could measure value added by randomly assigning students who are 
recommended to take developmental courses either to take the developmental courses or to 
enroll directly in traditional college-level courses instead. The difference between the success 
rates of the two groups would reflect the benefit, if any, of taking developmental courses. For 
several reasons, of course, this kind of experiment is unlikely to be done. 

An alternative approach to estimating the benefit of developmental coursework 
(Perkhounkova, Noble, & Sawyer, 2005) is based on operational data from course placement 
systems. With this method, we compare developmental students’ conditional probability of 
success, given test scores, with the corresponding conditional probability of success of non- 
developmental students with similar test scores. 

1. First, estimate the conditional probability of success from the test score and outcome data 
of students who first take a developmental course before taking the associated higher- 
level course. This conditional probability of success PDev(x) is a function of the test score 
x. 

2. Then, estimate the conditional probability of success from the test score and outcome 
data of non-developmental students (students who enroll directly in the associated higher- 
level course). This conditional probability of success P^ on Dev(x) is also a function of the 
test score x. 

3. Compare the estimated conditional probability of success function of developmental 
students, P Dev (x), to the estimated conditional probability of success function of non- 
developmental students, PmnDevix), at the actual test scores of developmental students. 
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If the developmental course is beneficial, then the conditional probability PDev(x) should be 
larger than the conditional probability PmnDevix) at most of the test scores x observed for the 
developmental students. In general, the difference between the two probabilities will vary by test 
score. This method therefore has the potential to identify which students will benefit most from 
taking a developmental course. 

Because students who take developmental courses typically have lower test scores than 
students who do not take developmental courses, this comparison requires a certain degree of 
extrapolation. If an institution applied cutoff scores rigidly, there would be complete 
extrapolation. We have found, however, that at most institutions there is considerable overlap in 
the test score distributions of developmental and non-developmental students, largely due to two- 
stage placement testing and/or advisors’ authority to waive course entry requirements. 

Course Data 

In this study, we analyzed data for six distinct pairs of courses: 

1 . Developmental English Composition and Standard English Composition 

2. Arithmetic and Elementary Algebra 

3. Elementary Algebra and Intermediate Algebra 

4. Intermediate Algebra and College Algebra 

5. Developmental Reading and American History 

6. Developmental Reading and Psychology 

Following the approach of Perkhounkova et al. (2005), we estimated, for each course pair, the 
conditional probability of success (defined in various ways), given test scores and other 
characteristics, for students who took the lower-level course (i.e., developmental course) before 
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taking the associated higher-level course . We then estimated the conditional probability of 
success for students who enrolled directly in the higher-level course. To address the first 
question posed by Phipps (1998), we also estimated the conditional probability of success in the 
lower-level course. 

Perkhounkova et al. (2005) showed that developmental instruction was effective only for 
students who earned a B or higher grade in the developmental course. We are not aware, 
however, of research related to pass/fail grades in developmental courses. In this study, we 
examined lower-level courses with A-F grades as well as lower-level courses with pass/fail 
grades. 

In principle, one could estimate conditional probabilities of success, given many other 
variables, in addition to test scores. Examples include background characteristics, high school 
coursework and grades, or psychosocial and situational variables. Conditioning on other 
variables would permit us to study particular groups of students, thereby yielding a more 
nuanced and accurate description of the benefit of developmental courses. Because of 
limitations in the data available for this study, we have conditioned only on test scores, part-time 
vs. full-time enrollment status, and college type (two-year vs. four-year). 

Data were not available on students’ academic goals, but enrollment in a two-year (vs. 
four-year) college, and part-time (vs. full-time) enrollment, might be considered surrogates of 
academic goals. Moreover, students who enroll in two-year colleges, and those who enroll in 
college part-time, have lower probabilities of persisting in college and completing a degree. 
Two- and four-year colleges also differ in the types of degrees they provide (i.e., Associate’s vs. 

2 For two of the three mathematics course pairs, the lower-level course and the higher-level course were both 
developmental courses. To avoid confusion, in this report the term “lower-level course” refers to the first course in 
each course pair, and the term “higher-level course” refers to the second course in each pair. The higher-level 
course could be either a developmental or college-level course. 
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Bachelor’s degrees). Furthermore, two-year colleges are perceived as being better equipped 
and/or less costly to provide developmental instruction (e.g., Ignash, 1997; Vandal, 2010; Shults, 
2000). We therefore statistically controlled for the type of institution (two-year vs. four-year) in 
which students initially enrolled and their first-year enrollment status (full-time vs. part-time). 

To provide additional perspective on students who take developmental courses, we also 
estimated the likelihood that enrolled students would take any developmental courses, as well as 
the number of developmental courses they would take. 

Data 

The data for this study consisted of the ACT Test student records and college outcomes 
data for 118,776 students who first enrolled in one of 75 postsecondary institutions. First-year 
entering cohort years ranged from 2002 to 2008 . The postsecondary institutions included two- 
year and four-year institutions from two states that explicitly use ACT Test scores for course 
placement 4 and from three four-year institutions from a third state that does not. All of the states 
are located in the south-central region of the United States. 

We used students’ ACT English, Mathematics, and Reading scores to predict later 
college outcomes. We did not use the ACT Science score as a predictor because course 
placement is most typically done in English, mathematics, and reading (the latter associated with 
placement in reading-intensive social science courses). 

Institutions and Enrollment Status 

Of the 75 institutions, 40 were two-year colleges and 35 were four-year colleges. In the 
analyses, we associated each student with the institution in which he or she first enrolled. We 
associated students who initially enrolled in more than one institution with the institution in 

3 The time span for follow-up data depended on the cohort year. Students and institutions were included only in 
analyses for which outcome data were available. 

4 ACT English, Mathematics, and Reading cutoff scores of 19. 
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which they completed the most terms, but included in the analyses each student’s entire 
academic record. Thus, for 72 of the 75 institutions, we were able to follow students who 
transferred to other in-state institutions. 

We also classified students by full-time or part-time enrollment status using credit hours 
attempted during their first fall and spring terms. We used credit hours earned if credit hours 
attempted was missing. We classified students with fewer than 24 total credit hours attempted 
during the first year as part-time, and those with 24 attempted hours or more as full-time. 

College Course Identification and Selection 

Institutions provided complete college transcripts for all their enrolled students. Using 
the course code list from ACT’s Course Placement Service® (ACT, 2012b) and the course 
catalogs for the institutions, we coded all courses as first-year vs. later, by level (developmental, 
standard college-level, or honors) and by whether the course was specific to a particular program 
or major (e.g., mathematics for elementary school teachers). We retained for analysis only the 
developmental or first-year college-level courses in English, mathematics, reading, and the social 
sciences that were not specific to a particular program or college major 5 . We calculated for each 
student the number of developmental courses taken within each subject area, as well as the 
number of times a given course was taken. 

We identified the courses with known sequencing (e.g., Arithmetic to Elementary 
Algebra or Developmental Reading to Psychology), and retained the most frequently occurring 
course sequences across institutions. We also required the selected courses to have data from at 
least 10 institutions. 

We excluded from the analyses students who skipped courses in the mathematics 
sequence. Moreover, if students took more than two mathematics courses in the mathematics 
5 We also excluded honors courses from the analyses. 
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course sequence, we used only the data for the first two courses, to avoid having intervening 
coursework influence test score-course outcome relationships. For example, if students took 
Arithmetic, Elementary Algebra, and Intermediate Algebra, we used only their data for 
Arithmetic and Elementary Algebra. 

We also required, for each pair of lower- and higher-level courses, that students took the 
lower-level course either before or concurrently with the first time they took the higher-level 
course. For students who took a lower-level course multiple times, we retained data from both 
the first time they took the course and from the last time they took the course prior to (or 
concurrently with) taking the higher-level course. 

For the higher-level course in a course pair, we retained data only from the first time 
students took the higher-level course. 

Course Grades 

Although most institutions reported grades using only one grading scale, others reported 
grades on both an A-F and a pass/fail (or satisfactory/unsatisfactory) scale. Thus, institutions 
could be included in both the A-F and pass/fail analyses. For all lower-level courses except 
Arithmetic, slightly more than half of the institutions using a pass/fail scale were two-year 
institutions. For Arithmetic, only one of 13 institutions using a pass/fail was a four-year 
institution. We recoded all pass/fail grades to a uniform pass/fail standard: “S,” “Credit,” “Pass,” 
and “P” were recoded to passing; “U,” “NC,” “NR,” and “NOT P” were recoded to failing 6 . 

We transformed A-F grades in the higher-level courses to two different levels of outcome 
variables: a B or higher grade (successful) vs. less than a B grade (unsuccessful), and a C or 
higher grade (successful) vs. less than a C grade (unsuccessful). For either level of outcome 

6 We also found grades that did not fit with either scale (e.g., audit, administrative withdrawal, etc.). We omitted 
these grades from the analyses. 
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variable, we classified withdrawals as unsuccessful. 7 We also classified students who took the 
lower-level course in a course pair, but did not take the higher-level course, as having an 

O 

unsuccessful outcome. 

For higher-level courses with pass/fail grades, we transformed the pass/fail grades to 
outcome variable levels according to the same method described in the preceding paragraph. 

For all lower-level courses, A-F or pass/fail grades from the last time the course was 
taken were retained in their original form as predictor variables for the analyses. A-F and 
pass/fail grades from the first time the course was taken were recoded to outcome variables and 
levels according to the same method described for higher-level courses. 9 
Longitudinal College Outcomes 

Each institution also provided up to six years of longer-term outcome data. The data 
included term-by-term credit hours attempted, credit hours earned, retention indicators, 
cumulative GPAs, and Associate’s or Bachelor’s degree completion indicators. From these 
variables, we coded the following binary outcome variables: 

• Earned 2.0 or higher Term 1 GPA and persisted to Term 2 at the same institution 

• Earned 3.0 or higher Term 1 GPA and persisted to Term 2 at the same institution 

• Earned 2.0 or higher Year 1 GPA and persisted to Year 2 at the same institution 

• Earned 3.0 or higher Year 1 GPA and persisted to Year 2 at the same institution 

• Earned 2.0 or higher Year 2 GPA and persisted to Year 3 at the same institution 

• Earned 3.0 or higher Year 2 GPA and persisted to Year 3 at the same institution 

7 The percentages of withdrawals in the higher-level courses ranged from 7% in Standard English Composition and 
in Psychology to 21% in Intermediate Algebra. 

8 This adjustment affected 1% or less of students in any course pair. 

9 W grades were included with A-F grades from the lower-level courses when recoded to success outcome variables 
and levels. The percentages of W grades ranged from 11% in Developmental English Composition to 22% in 
Elementary Algebra. W grades were not included as predictors of success in higher-level courses or of other college 
outcomes. 
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• Earned 2.0 or higher GPA at time of degree completion or at last term enrolled (if degree 
was not completed) 

• Earned 2.5 or higher GPA at time of degree completion or at last term enrolled (if degree 
was not completed) 

• Earned 3.0 or higher GPA at time of degree completion or at last term enrolled (if degree 
was not completed) 

• Earned Associate’s degree within three years (students whose first institution was two- 
year) 

• Earned Bachelor’s degree within five years (students whose first institution was two- or 
four-year) 

• Earned Bachelor’s degree within six years (students whose first institution was two- or 
four-year) 

The last three outcomes pertain to degree completion within specified time periods. For the 
years spanning the data for this study, the Integrated Postsecondary Education Data System 
(IPEDS) standard time frame for evaluating graduation rates was 150% of normal time. This 
corresponds to Associate’s degree completion in three years and Bachelor’s degree completion in 
six years. IPEDS now collects graduation rates from institutions at 100%, 150%, and 200% of 
normal time. 

Limitations of the Data 

The data for this study consisted of ACT-tested college students who were enrolled in 
two- and four-year institutions, mostly from two south-central states. These two states use ACT 
English, Mathematics, and Reading scores in course placement; the two-year colleges in these 
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states also use ACT’s COMPASS® tests (ACT, 2012c) for course placement 10 . Although a large 
proportion of high school graduates in both states take the ACT Test, it is not required for two- 
year college admission. Further research is planned that will focus on ACT COMP ASS-tested, 
two-year college students, to help determine the effectiveness of developmental instruction for 
the broader pool of students enrolled in two-year colleges. ACT COMPASS data could also 
assist in further differentiating the academic preparedness of students in the three lower-level 
mathematics courses. 

We focused on explicitly defined pairs of lower-level and higher-level courses in this 
research. Developmental education does not operate in a vacuum, however. We had no 
information about students’ participation in other educational support programs or student 
services (e.g., learning communities, tutoring, etc.). Moreover, in using particular course pairs, 
we did not study the content or level of other coursework taken, either within or across subjects. 
Either of these conditions could have influenced students’ success in the higher- level course or 
later in college. 

The sample for this study represented ACT-tested enrolled students, rather than all 
students who enrolled in these two- and four-year institutions. As such, the students in our 
sample were more likely to be college-bound and traditional-aged (17-19 age bracket). 

This study compares college outcomes for students who did and did not first take a 
developmental course. The extent to which these two groups differ on characteristics not 
accounted for in the models could affect the results. For example, several studies (e.g., Boylan, 
1995; Ignash, 1997) noted that students enrolled in developmental coursework are more likely to 
be nontraditional students with economic hardship (Bettinger & Long, 2007). These students 

10 ACT COMPASS is a computer-adaptive college placement testing program that evaluates students' current 
knowledge and skills in Reading, Writing Skills, Writing Essay, and Mathematics. It also includes tests for placing 
students with limited English proficiency in appropriate courses. 
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might benefit differently from developmental coursework than traditional-age students do. 
Differences on other student characteristics such as race/ethnicity, family income, high school 
GPA, or behavior might also affect the results. Because some within-institution sample sizes for 
particular courses are small, and because data on potential covariates were incomplete, we did 
not include the covariates in the models for this study. We hope to do so in future research. 

We examined the race/ethnicity, family income, educational plans, and high school GPAs 
of similar students who took the lower-level courses in this study and those who enrolled directly 
in the higher-level courses. The comparisons are displayed in Table 1 (for students who took 
developmental reading or writing courses) and in Table 2 (for students who took developmental 
mathematics courses). For comparability purposes, students in the higher-level courses were 
limited to those who scored at or below the 95 th percentile of the relevant ACT Test scores of 
students who took the lower-level course. 11 The underrepresented minority group in Tables 1 
and 2 includes African American, American Indian/ Alaskan Native, and Hispanic students. 


11 Score values for the course pairs were 17 for Standard English Composition (Developmental English 
Composition/Standard English Composition), 16 for Elementary Algebra (Arithmetic/Elementary Algebra), 17 for 
Intermediate Algebra (Elementary Algebra/Intermediate Algebra), 18 for College Algebra (Intermediate 
Algebra/College Algebra), 20 for American History (Developmental Reading/American History), and 19 for 
Psychology (Developmental Reading/Psychology). 
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Table 1 

Demographic Characteristics of Lower-Scoring Students who took Lower- or Higher-Level 
Courses in English and Reading Course Pairs 



Dev. 

English 

Comp. 

Std. 

English 

Comp. 

Dev. 

Reading 

Am. 

History 

Dev. 

Reading 

Psych. 

HS core curriculum 

Taken 

.54 

.55 

.54 

.61 

.54 

.59 

Not taken 

.38 

.38 

.38 

.33 

.38 

.34 

Missing 

.08 

.07 

.08 

.07 

.08 

.07 

Race/ethnicity 

Underrep. minority 

.38 

.37 

.42 

.30 

.40 

.32 

White 

.53 

.54 

.51 

.61 

.51 

.59 

Missing 

.09 

.09 

.07 

.08 

.08 

.09 

Family income 

< $30,000 

.39 

.33 

.37 

.29 

.39 

.31 

$30,000460,000 

.28 

.30 

.28 

.31 

.27 

.31 

$60,0004100,000 

.11 

.14 

.12 

.16 

.12 

.15 

> $100,000 

.04 

.04 

.04 

.05 

.03 

.05 

Missing 

.18 

.18 

.19 

.18 

.18 

.18 

Educational plans 

2-year college 

.08 

.06 

.07 

.04 

.08 

.04 

4-year degree or more 

.73 

.78 

.74 

.81 

.74 

.80 

Other 

.07 

.06 

.08 

.05 

.07 

.05 

Missing 

.11 

.12 

.11 

.11 

.11 

.11 

High school GPA 

Mean 

2.90 

3.08 

2.93 

3.21 

2.91 

3.16 

Proportion missing 

.18 

.17 

.17 

.15 

.17 

.16 

ACT subject area score 

Mean 

14.5 

14.8 

15.1 

16.8 

15.1 

16.0 
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Table 2 

Demographic Characteristics of Lower-Scoring Students who took Lower- or Higher-Level 
Courses in Mathematics Course Pairs 



Arith. 

El. 

Algebra 

El. 

Algebra 

Int. 

Algebra 

Int. 

Algebra 

Col. 

Algebra 

HS core curriculum 

Taken 

.49 

.48 

.52 

.60 

.64 

.64 

Not taken 

.42 

.43 

.40 

.32 

.30 

.29 

Missing 

.09 

.09 

.08 

.08 

.07 

.06 

Race/ethnicity 

Underrep. minority 

.36 

.42 

.42 

.41 

.34 

.27 

White 

.56 

.49 

.49 

.51 

.58 

.65 

Missing 

.08 

.08 

.08 

.08 

.08 

.08 

Family income 

< $30,000 

.46 

.43 

.40 

.37 

.33 

.26 

$30,000-$60,000 

.25 

.26 

.28 

.29 

.31 

.33 

$60,0004100,000 

.09 

.10 

.12 

.13 

.14 

.17 

>$100,000 

.02 

.03 

.03 

.04 

.04 

.06 

Missing 

.18 

.19 

.18 

.17 

.17 

.18 

Educational plans 

2-year college 

.13 

.08 

.08 

.05 

.05 

.03 

4-year degree or more 

.67 

.73 

.74 

.78 

.79 

.82 

Other 

.10 

.07 

.07 

.06 

.06 

.05 

Missing 

.10 

.12 

.11 

.11 

.10 

.10 

High school GPA 

Mean 

2.77 

2.82 

2.92 

3.00 

3.10 

3.27 

Proportion missing 

.18 

.20 

.18 

.17 

.16 

.13 

ACT subject area score 

Mean 

15.0 

14.8 

15.6 

16.0 

16.7 

16.8 


Students who took lower-level courses were, in general, more likely to have families with 
incomes in the lowest income range, have a lower high school GPA, and have a lower ACT Test 
subject area score; and were less likely to plan to complete a Bachelor’s degree or more. 
Racial/ethnic differences were found for two of the three mathematics course pairs and for both 
Developmental Reading/ American History and Developmental Reading/Psychology. For all but 
one of these course pairs, students taking the lower-level course were more likely to be from an 
underrepresented minority group. For the Arithmetic/Elementary Algebra course pair, however, 
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students taking only Elementary Algebra were more likely to be from an underrepresented 
minority group. 

In one state, data on degree completion extended only through year five, thereby 
preventing conclusions about six-year degree completion for institutions in that state. As a 
result, we could examine six-year Bachelor’s degree completion rates for only a maximum of 22 
institutions. 

Method 

Descriptive Statistics 

We calculated sample sizes and means (or proportions), pooled across institutions, for 
each pair of courses studied. This information is reported in Appendix A. 

Participation in Developmental Courses 

We first modeled the probability of a student taking any developmental coursework in 
English, mathematics, and/or reading, given his or her corresponding ACT Test score. We 
calculated a variable indicating whether a student took any developmental course in English, 
mathematics, or reading; it was not limited to the six courses identified for this study. 

We next predicted the total number of developmental courses a student would take in 
each subject area, given his or her corresponding ACT Test score. The outcome variables in this 
analysis included repeats of the same developmental course. Across the three subject areas, the 
maximum number of developmental courses taken ranged from four in reading to six in 
mathematics. 

We estimated hierarchical logistic regression models to predict these variables (see 
discussion below), with ACT English, Mathematics, or Reading scores as predictors. For 
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predicting the number of courses taken, we estimated hierarchical linear regression models and 
included a quadratic term for the ACT Test score to improve model fit. 

Evaluating the Success of Developmental Students 

We estimated hierarchical logistic regression models (described below) for predicting 
students’ chances of a B or higher, C or higher, or a passing grade in each lower- level course the 
first time they took it. These models describe the relationship between predictor variables and 
course outcomes and account for variation in these relationships across institutions. The 
predictor variables were the relevant ACT Test score, full-time/part-time enrollment status, and 
the interaction (product) of ACT Test score with full-time/part-time enrollment status. 

Next, we estimated hierarchical logistic regression models for students who enrolled in a 
higher- level course after first taking the associated lower-level course. To predict the 12 
outcomes described previously, we used the same variables as in the lower-level course models. 
We also estimated models with the following additional predictor variables: the grade students 
received in the lower-level course, the interaction of lower-level course grade with full-time/part- 
time enrollment status, and the interaction of lower-level course grade with the relevant ACT 
Test score. 

We developed separate models for students receiving A-F grades and for those receiving 
pass/fail grades in the lower- level course. In general, both the student sample sizes and the 
numbers of institutions for lower-level courses with pass/fail grades were much smaller than 
those with A-F grades. As a result, the analyses for these particular courses were often 
constrained. 

Finally, we estimated models for students who enrolled directly in a higher-level course 
without taking the associated lower-level course. The predictor variables were the relevant ACT 
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Test score, full-time/part-time enrollment status, and the interaction of ACT Test score with full- 
time/part-time enrollment status. 

Hierarchical logistic regression. Logistic regression models the relationship between 
the probability p of a successful outcome and one or more predictor variables. In the case of a 
single predictor X with observed value x, the logistic model is: 

In [ p(x)/(l - j9(X))] = a + bx , 

where a and b are regression coefficients (weights) and In is the natural logarithm function. In 
hierarchical logistic regression, the intercept coefficient a and the slope coefficient b potentially 
vary randomly across institutions. This property reflects the natural clustering of students within 
institutions. The hierarchical model estimates both fixed effects (the average regression 
coefficients across institutions) and random effects (the variability of the regression coefficients 
across institutions). 

In this study, we estimated random slope and random intercept models when the 
estimated variability in the slopes or the intercepts across institutions was significantly different 
from zero (p > .01). We also used the institution-specific mean values for each predictor 
variable, as well as institution type (two-year vs. four-year), to predict outcomes at the institution 
level. All student and institutional predictor variables (except institution type) were grand-mean 
centered. 

Results 

Descriptive Results 

Tables A-l through A-6 in Appendix A contain pooled descriptive statistics for the six 
course pairs. Each table contains the number of institutions, the number of students, and the 
means (or proportions) for all predictor and outcome variables. These statistics are presented 
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separately for students who took the lower-level course before taking the associated higher-level 
course, and for those who enrolled directly in the higher-level course. The statistics are also 
presented separately according to the grading scale in the lower-level course (A-F or pass/fail). 

From the “No. of students” columns in Tables A-l through A-6, we can calculate the 
percentage of students who took the lower-level course before taking the associated higher-level 
course. This percentage ranged from 6% for Developmental Reading/American History to 31% 
for Elementary Algebra/Intermediate Algebra. Among the students who did take the lower-level 
course, the percentage who earned an A-F grade (vs. a pass/fail or other grade) also varied 
widely: It ranged from 55% (Developmental Reading/ American History) to 95% (Elementary 
Algebra/Intermediate Algebra). 

Compared to ACT-tested enrolled freshmen nationally, the students in our sample (even 
those who enrolled directly in standard college-level courses) had lower average ACT Test 
scores. The national average ACT Composite score of enrolled freshmen in 2005-06 (ACT, 
2006) was 21.9, with a standard deviation of 4.7. The average Composite scores of students in 
the sample who enrolled directly in standard first-year college courses (Standard English 
Composition, College Algebra, American History, and Psychology) ranged from 20.9 to 21.4. 
As one would expect, students in Developmental English Composition, Arithmetic, and 
Developmental Reading had much lower average Composite scores (ranging from 15.6 to 18.0). 
However, students who received A-F grades in lower-level English and mathematics courses 
typically had somewhat higher average ACT Test scores than those who received pass/fail 
grades. 12 For Developmental Reading (taken before either American History or Psychology), the 
opposite was true. 

12 The exception was Elementary Algebra/Intermediate Algebra, for which average ACT Test scores were similar 
for the two groups. 
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In general, students who took lower-level courses before taking higher-level courses were 
more likely to have enrolled part-time than were students who took only the higher-level courses 
(by .06 to .38), and were less likely to enroll in a four-year college (by .08 to .19). Students who 
received pass/fail grades in the lower- level course were much more likely to have enrolled full- 
time (by .33 to .60) than those who received A-F grades and, with the exception of Arithmetic, 
were more likely to have enrolled in a four-year institution (by .08 to .12). 

Across the hierarchical sequence of mathematics courses, course level related strongly to 
the proportion of students enrolled full-time and to the proportion of students enrolled at a four- 
year institution: The proportion of full-time students ranged from .24 for Arithmetic to .66 for 
Intermediate Algebra, and the proportion enrolled at a four-year institution ranged from .20 to 
.37 for the same courses. 

An important consideration in using lower-level course grades in predictive models, 
either as predictors or outcome variables, is whether they have sufficient variability. All of the 
lower-level courses that assigned pass/fail grades had extremely high pass rates (.94 - .98). 
These high pass rates limited the analyses that could be done with pass/fail grades: Moreover, we 
could not estimate models for many of the long-term outcomes because none of the students who 
received a “Pass” grade in a lower-level course achieved a successful outcome later in college. 

Students who take developmental courses in college have lower Associate’s and five- and 
six-year Bachelor degree completion rates than students who take only higher-level courses, 
possibly because developmental courses typically do not count towards a degree. As shown in 
Tables A-l through A-6, this was generally the case for almost all course pairs. The exceptions 
were Elementary Algebra/Intermediate Algebra (all three degree outcomes) and 
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Arithmetic/Elementary Algebra (Associate’s degree). We did not have sufficient data to examine 
five-year degree completion rates for students who took Arithmetic. 

It is worth noting, however, that students who took Developmental English Composition 
or Developmental Reading courses completed their Bachelor’s degree in six years at rates 
comparable to the five-year completion rates of non-developmental students. For example, the 
six-year Bachelor’s degree completion rate for Developmental English students was .36, and the 
five-year Bachelor’s degree completion rate for Standard English Composition students was .37. 
Thus, students who took lower-level courses eventually completed their Bachelor’s degrees at a 
rate similar to that of students who enroll directly in the corresponding higher-level courses, but 
they required more time to do so. 

Taking Developmental Coursework 

Across all students in the sample, 19% took one or more developmental courses in 
English, 37% took one or more developmental courses in mathematics, and 6% took one or more 
developmental courses in reading. These values were based on any developmental English, 
mathematics, or reading course included in a student’s transcript, not just the six courses 
identified for this study. The percentages for English and mathematics were somewhat higher 
than those reported nationally (14% and 22%, respectively; Parsad & Lewis, 2003); the 
percentages for reading were slightly lower (1 1%). 

Figure 1 shows the probability of taking any developmental English, mathematics, or 
reading coursework, given ACT Test score. The probabilities associated with the state cutoff 
score of 19 on ACT English, Mathematics, and Reading are also shown in the figure. The 
circled portions of the lines represent extrapolations. 
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Figure 1. Probability of taking developmental English, mathematics, or reading, given ACT Test 
score. 

The probability of taking developmental coursework was strongly and inversely related 
to ACT Test score. Students with an ACT Mathematics score of 19 had a .29 chance of taking 
developmental mathematics coursework, as compared to a .77 chance for those with a 
Mathematics score of 15. The corresponding probabilities for ACT English scores of 19 and 15 
were .15 and .41, respectively; the probabilities for ACT Reading scores of 19 and 15 were .03 
and .10, respectively. Thus, ACT English, Mathematics, and Reading score values lower than 
the state cutoffs corresponded to greater chances of taking a developmental course in those 
subjects, but not to certainty in taking a developmental course. This finding could be attributed 
to the fact that students were allowed to take a second test at college entry to demonstrate their 
preparation for higher-level coursework. 
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Figure 2 illustrates the expected number of developmental courses that students took, 
given their ACT Test scores. These results are based on the data of students who took at least 
one of the five developmental courses in the study, and are intended to illustrate the likelihood of 
students taking developmental courses multiple times, given their ACT Test score. The solid 
lines reflect results based on observed data (middle 90%); the circled portions of the lines 
represent extrapolations. 



Figure 2. Expected number of times taking developmental English, mathematics, or reading 
courses, given ACT Test score, among students who took at least one developmental course in a 
subject. 


The expected number of English and reading developmental courses taken was 
approximately 1, regardless of test score. There was a much stronger relationship, however, 
between the number of developmental mathematics courses taken and ACT Mathematics score. 
For students with Mathematics scores of 17 or lower, each 1 point decrease in score was 
associated with an increase of nearly . 1 in the expected number of developmental mathematics 
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courses taken. Moreover, it is worth noting that some students with an ACT Mathematics score 
as high as 20 took at least one developmental mathematics course, and some would be expected 
to take more than one developmental mathematics course. 

Models for Predicting Success in College 

Table B-l in Appendix B summarizes the six hierarchical logistic regression models for 
predicting success in lower-level courses. This table contains both the fixed effects (the 
estimated average regression coefficients across all institutions) and the variance components 
(estimates of the variance of the regression coefficients across institutions). 

Tables B-2, B-4, B-6, B-8, B-10, and B-12 summarize the fixed effects of the models for 
predicting success in college (e.g., success in the associated higher-level course, retention/GPA, 
and degree completion). Each of these tables contains separate models for the following four 
student groups: 

• All students who took a lower-level course before taking the associated higher-level 
course 

• Students who took a lower-level course with an A-F grade scale before taking the 
associated higher-level course 

• Students who took a lower-level course with a pass/fail grade scale before taking the 
associated higher-level course 

• Students who enrolled directly in the higher-level course. 

Each table shows, for every outcome variable, the institution-level and student-level 
coefficients. The coefficients that were not statistically significant are shaded. We removed 
college type and institution- and student-level interaction terms from the models when they were 
not statistically significant. We also removed institution-level coefficients for average lower- 
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level course grade and/or proportion full-time that were not statistically significant and were 
highly collinear with the intercept. ACT Test score, enrollment status, and lower-level course 
grade were retained in all models, regardless of statistical significance, as they were primary 
predictor variables in the study. 

Tables B-3, B-5, B-7, B-9, B-ll, and B-13 summarize the variance components 
associated with the fixed effect coefficients (intercept and slope) in the models. These tables are 
structured similarly to Tables B-2, B-4, B-6, B-8, B-10, and B-12. 

Estimated Probabilities of Success 

The fixed effects in Tables B-2, B-4, B-6, B-8, B-10, and B-12 can be used to calculate 
estimated probabilities of success at typical institutions. Distributions of probabilities for each 
course pair and outcome are summarized for all students and by selected course grades in 
Appendix C. In this section we illustrate and discuss the estimated probabilities for different 
outcome variables and under different scenarios. 

Full-time/part-time enrollment status was an important predictor in most of the models. 
To simplify and condense the discussion, we focus on the college outcomes of full-time students 
only in the next sections. Comparable results for part-time students may be obtained from the 
first author. Following the discussion of the results for full-time students, we then report on 
typical differences in the results between part- and full-time students. 

Developmental course outcomes for full-time students, by ACT Test score. Among 
the lower- level courses that used an A-F grade scale, ACT Test scores were strongly related to 
course success: For all courses, higher ACT Test scores corresponded to a greater probability of 
earning a B or higher grade. This result was also true for achieving a C or higher grade in all 
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lower-level courses (though the coefficient for ACT Mathematics score was not statistically 
significant for Arithmetic). 

We obtained different results for the lower-level courses that used a pass/fail grade scale. 
ACT Test score was not associated with a “pass” grade in any of the lower- level courses, except 
for Arithmetic and Intermediate Algebra. These results are likely attributable to the very high 
“pass” rates and the small sample sizes for these courses. College type was not a statistically 
significant institution-level predictor for any of these courses and was therefore removed from 
these models. 

Figures 3-5 illustrate the probabilities of success for each lower-level course and 
outcome. For all lower-level courses, students had at least a .86 probability of achieving a C or 
higher grade the first time they took the course, and at least a .93 probability of achieving a 
passing grade. Moreover, for all courses except Elementary Algebra, students in these courses 
had a greater than .50 probability of a B or higher grade in the course, irrespective of their ACT 
Test score. Elementary Algebra was the most difficult course for achieving a B or higher grade; 
the highest observed score (17) was associated with only a .70 probability of achieving a B or 
higher grade, as shown in Figure 4. 
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Figure 3. Probabilities of success in Developmental English Composition, first time taken, by 
ACT English score. 



Figure 4. Probabilities of success in Arithmetic, Elementary Algebra, and Intermediate Algebra, 
first time taken, by ACT Mathematics score. 
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Figure 5. Probabilities of success in Developmental Reading (prior to taking either American 
History or Psychology), first time taken, by ACT Reading score. 

Higher-level course outcomes for full-time students, by ACT Test score. As shown 
in Appendix B, ACT Test scores were positive and statistically significant predictors of success 
in all of the higher-level courses. This result pertains both to students who first took the lower- 
level course and to those who enrolled directly in the higher-level course. 

For Standard English Composition, Elementary Algebra/Intermediate Algebra, 
Developmental Reading/ American History, and Developmental Reading/Psychology, students 
who first took the lower-level course before taking the higher-level course had similar or lower 
probabilities of success in the higher-level course than would have been expected had they 
enrolled directly to that course. This result was true of both the B-or-higher and the C-or-higher 


success criteria. 
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Figures 6, 7, and 8, and Appendix C illustrate this finding. In these and subsequent 
figures, solid and dashed lines reflect the middle 90% of observed scores. The solid lines show 
estimated probabilities of success for students who enrolled directly in the higher-level course. 
The dashed lines show estimated probabilities of success for students who first took the lower- 
level course. Circles on the solid lines represent extrapolation to scores below those of students 
who enrolled directly in the higher-level course. 



ACT English score 

Figure 6. Probability of success in Standard English Composition with and without taking 
Developmental English Composition. 


As shown in Figure 6, students with ACT English scores between 9 and 17 had a .75 or 
higher estimated probability of a C or higher grade in Standard English Composition, had they 
enrolled directly in the course. If they took Developmental English Composition first, however, 
they had a lower estimated probability of a C or higher grade (.69 or higher). A similar result 
occurred for earning a B or higher grade in Standard English Composition. 
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We found a similar result for Elementary Algebra/Intermediate Algebra (Figure 7). 
Students who first took Elementary Algebra had lower probabilities of either a B or higher grade 
or a C or higher grade in Intermediate Algebra than would be expected, had they enrolled 
directly in Intermediate Algebra. For the C or higher outcome, the differences between the two 
student groups in their probability of success increased slightly as ACT Mathematics score 
increased. 



Figure 7. Probability of success in Intermediate Algebra with and without taking Elementary 
Algebra. 

The results for the Developmental Reading/ American History and Developmental 
Reading/Psychology student groups were similar; the results for Developmental 
Reading/American History are shown in Figure 8. Students who first took Developmental 
Reading had a lower probability of obtaining either a B or higher grade or C or higher grade in 
American History, regardless of their ACT Reading score, than students who enrolled directly in 
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the higher-level course. For the B or higher outcome, differences in probability of success 
between the two student groups increased markedly as ACT Reading score increased. For 
students with very low scores, the differences between groups in the probability of a B or higher 
grade were small. 



ACT Reading score 

Figure 8. Probability of success in American History with and without taking Developmental 
Reading. 

In contrast, there were positive results for other course pairs. As shown in Figure 9, 
students who first took Arithmetic had a higher probability of success in Elementary Algebra 
than would be expected, had they enrolled directly in Elementary Algebra. Students who took 
Arithmetic had consistently higher probabilities of success than students who enrolled directly in 
Elementary Algebra (by .03 to .09 for C or higher and .05 to .06 for B or higher). It is also 
interesting to note that as ACT Mathematics score increased, differences increased slightly 
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between the two groups’ probabilities of a B or higher grade, and decreased between the two 
groups’ probabilities of a C or higher grade. 



Figure 9. Probability of success in Elementary Algebra with and without taking Arithmetic. 


The probability of success in College Algebra of students who first took Intermediate 
Algebra was slightly higher than that of students who enrolled directly in the higher-level course. 
As shown in Figure 10, the probability of either a B or higher grade or a C or higher grade in 
College Algebra of students who first took Intermediate Algebra was slightly higher than that of 
students who enrolled directly in the higher-level course (by .02 to .05). 
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Figure 10 . Probability of success in College Algebra with and without taking Intermediate 
Algebra. 


Lower-level course grades as predictors of success in higher-level courses. Across all 
course pairs, the probability of success in the higher-level course for students who took the 
lower-level course also depended on the A-F grade they received in the lower-level course. The 
pass/fail grade in the lower-level course was, for almost all course pairs, not a statistically 
significant predictor of higher-level course success . 13 

For all course pairs except those including Developmental Reading, the probability of 
earning a B or higher grade in the higher-level course also depended on ACT Test score, even 
when lower-level course grade was included in the models. ACT Reading score was not a 


13 The exceptions were Developmental English Composition/Standard English Composition (both B or higher and C 
or higher outcomes) and Developmental Reading/Psychology (C or higher outcome only). We could not even 
develop higher-level course models using pass/fail grades for Arithmetic/Elementary Algebra and Elementary 
Algebra/Intermediate Algebra. 
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statistically significant predictor of success in American History or Psychology after 
Developmental Reading grade was added to the models. 

In the analysis of Developmental English Composition/Standard English Composition, 
Developmental Reading/American History, and Developmental Reading/Psychology, only 
students who received an A grade in the lower-level course had a higher probability of success in 
the higher-level course than would be expected, had they not taken the lower-level course. 
Figure 1 1 illustrates this result for Developmental Reading/Psychology (C or higher grade; see 
also Table C-6 in Appendix C). 



ACT Reading score 

Figure 11. Probability of C or higher grade in Psychology by Developmental Reading grade and 
ACT Reading score. 


In Figure 1 1, the estimated probability of a C or higher grade in Psychology ranged from 
.67 to .91 for students who enroll directly in the course (for ACT Reading scores of 11 to 31). 
The corresponding probabilities of success for students who first enroll in Developmental 
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Reading were .80, .70, and .57 for grades of A-C, respectively. Therefore, a grade of A was the 
only grade where the probabilities of success in Psychology always exceeded those for students 
who enroll directly in Psychology (with decreasing benefits as ACT Reading score approached 
19). 

For most mathematics course pairs, only A or B grades in the lower-level course 
corresponded to higher probabilities of success in the higher-level course than would be 
expected, had students enrolled directly in the higher-level course. This result occurred for both 
the B or higher and C or higher outcome levels (with the exception of the C or higher outcome 
for the Arithmetic/Elementary Algebra course pair). Figure 12 below illustrates this result for B 
or higher grades in Elementary Algebra, given ACT Mathematics score and grade in Arithmetic. 
A and B grades in Arithmetic were associated with substantially higher probabilities of B or 
higher grades than would be expected for similar students who enrolled directly in Elementary 
Algebra. C grades in Arithmetic were associated with virtually no increase in the probability of 
B or higher grades in Elementary Algebra, but were associated with increases in the probability 
of a C or higher grade. 
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Figure 12. Probability of B or higher grade in Elementary Algebra by Arithmetic grade and ACT 
Mathematics score. 


For Elementary Algebra/Intermediate Algebra and Intermediate Algebra/College 
Algebra, a lower-level course grade of B was associated with higher probabilities for those with 
scores of 17 or lower. The difference in probabilities decreased as ACT Mathematics score 
increased. The results for Intermediate Algebra/College Algebra are shown in Figure 13 for the 
B or higher success criterion. 
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Figure 13. Probability of B or higher grade in College Algebra by Intermediate Algebra grade 
and ACT Mathematics score. 


Early college outcomes of full-time students. We next discuss outcomes related to the 
Term 1, Year 1, and Year 2 success of full-time students according to their lower-level 
coursework 14 . 

For most course pairs, taking the lower-level course was associated with substantially 
higher probabilities of a Term 1 GPA of 2.0 or higher, or 3.0 or higher, and persisting to Term 2 
than would be expected, had students enrolled directly in the associated higher-level course. 
Figure 14 illustrates this result for Intermediate Algebra/College Algebra. 


14 Lower- and higher-level coursework could have been taken before, during, or after the first fall term. Lower-level 
courses taken the last time before the higher-level course were taken the first fall or winter term, or summer terms 
just prior, by 84% (Intermediate Algebra) to 96% (Arithmetic) of students. Higher-level courses were taken during 
this time period by 76% (American History) to 95% (Standard English Composition) of students. 
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Figure 14. Probability of a Term 1 cumulative GPA of 2.0 or 3.0 or higher and persisting to 
Term 2, for students who did and did not take Intermediate Algebra before College Algebra. 

Figure 14 also illustrates another result: For students who enroll directly in the higher- 
level course, ACT Test score was almost always a statistically significant predictor of early 
college success. (Note the steep slopes of the solid lines.) In contrast, ACT Test score was not a 
statistically significant predictor of early college success for students who first enroll in the 
lower-level course. (Note that the dotted lines are nearly flat.) This result is what one should 
expect: Placement decisions for most students in the study were based on ACT Test scores, 
resulting in restriction in the range of ACT test scores for students in the lower-level courses. 
Moreover, if lower-level coursework is effective in providing the knowledge and skills that are 
necessary for success in the next course, then test scores obtained before taking lower-level 
coursework no longer reflect what students know and are able to do, once they take the 


coursework. 


40 


The benefit of taking lower-level courses for improving students’ early college success 
gradually decreased over time and, in some cases, disappeared. Figure 15 illustrates this result 
for Intermediate Algebra/College Algebra. Note that the probabilities of success for students who 
enroll in Intermediate Algebra were higher than those for students who enroll directly in College 
Algebra, but the differences between the two student groups were smaller than those shown in 
Figure 14. 



Figure 15. Probability of a Year 1 cumulative GPA of 2.0 or 3.0 or higher and persisting to Year 
2 for students who did and did not take Intermediate Algebra before College Algebra. 


With respect to the Year 2 cumulative GP A/persistence outcomes, the probability of 
success of students who first took the lower-level course in each course pair was generally 
comparable to the probability of students who enrolled directly in the associated higher-level 
course. For most lower-level courses, the benefit to Year 2 cumulative GP A/persistence also 
depended on ACT Test score, but was small nonetheless 
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Lower-level course grades as predictors of early college outcomes of full-time students. 
Grades as low as a D in Developmental English Composition or Arithmetic were associated with 
a higher probability of success in Term 1 than would be expected, had lower-scoring students 
enrolled directly in the higher-level course. Grades of A and B, and occasionally C (depending 
on ACT Test score), in the lower-level course were associated with higher probabilities of Term 
1 success for the other developmental courses. 

For Year 1 college outcomes (Year 1 cumulative GP A/persist to Year 2), grades of A and 
B in the lower-level course were almost always associated with a higher probability of success 
than would be expected, had students enrolled directly in the higher-level course. For some 
lower-level course/outcome combinations, however, the benefit of a B grade depended on ACT 
Test score. 

For one-third of the course pair/outcome level combinations for Year 2 success (Year 2 
GP A/persist to Year 3), only a grade of A in the lower-level course was associated with a higher 
probability of success than would be expected, had students enrolled directly in the higher-level 
course. Figure 16 shows the results for Psychology and the Year 2 2.0 GPA outcome. For 
almost all of the other lower-level courses and outcomes, grades of A and B in the lower-level 
course were associated with higher probabilities of success. 
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ACT Reading score 

Figure 16. Probability of a Year 2 GPA of 2.0 or higher and persisting to Year 3 for students 
who took Psychology, by Developmental Reading grade. 

ACT Mathematics score had a statistically significant negative weight in some of the 
joint models with Elementary and Intermediate Algebra grade for predicting Year 1 and/or Year 
2 outcomes. Further investigation pointed to the transfer of two-year college students to four- 
year institutions as the likely cause. First, the negative weights occurred only for two-year 
institutions. Moreover, we modified the retention component of the success criteria to include 
transfer to other institutions (instead of retention at the same institution). When we re-estimated 
the models with the new criteria, the statistically significant negative weights disappeared. 

Cumulative GPA at graduation/last term enrolled for full-time students. In this 
section we examine outcomes defined by cumulative GPA at graduation/last term. As in 
previous analyses, we compared students who did and did not take a lower-level course before 
taking the associated higher-level course in each course pair. 


43 


For students who enrolled directly in the higher-level course, ACT Test score was almost 
always a statistically significant predictor of cumulative GPA at graduation/last term. (The sole 
exception was students who enrolled directly in Intermediate Algebra when predicting a 2.0 or 
higher GPA). In contrast, ACT Test score was not a statistically significant predictor of GPA at 
graduation/last term in the joint models that included lower-level course grade 15 . For these 
students, the grade in the lower-level course predicted GPA at graduation. This finding is 
consistent with other ACT research studies that have shown that the effect of ACT Test score 
diminishes or disappears once first-year grades are considered (e.g., Allen & Robbins, 2010; 
Radunzel & Noble, 2012). 

For most course pairs, the probabilities of a GPA at graduation/last term of 2.0, 2.5, or 
3.0 for students who first enroll in the lower-level course were similar to or lower than those that 
would be expected, had they enrolled directly in the higher-level course. Figure 17 illustrates 
this result for Developmental English Composition/Standard English Composition. Note that the 
probabilities were increasingly disparate between the two student groups as ACT English score 
increased. 


15 The exceptions were models for students who took Developmental English Composition, Arithmetic, and 
Developmental Reading for a pass/fail grade before taking either American History or Psychology. 
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ACT English score 

Figure 17. Probability of a GPA at graduation/last term of 2.0, 2.5, or 3.0 or higher for students 
who did and did not take Developmental English Composition before Standard English 
Composition. 


We found a small benefit to GPA at graduation/last term for students who took 
Arithmetic before Elementary Algebra (see Figure 18). Differences in probabilities were slight 
(.07 or less; see Table C-2) between the two student groups at all three GPA levels. However, at 
the 3.0 GPA level, students who first took Arithmetic had increasingly greater probabilities of 
success as ACT Mathematics score increased, compared to students who enrolled directly in 
Elementary Algebra. 
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Figure 18. Probability of a GPA at graduation/last term of 2.0, 2.5, or 3.0 or higher for students 
who did and did not take Arithmetic before Elementary Algebra. 


Degree completion of full-time students. In this section we examine degree 
completion, including completing an Associate’s degree in three years or completing a 
Bachelor’s degree in five or six years. For students who enrolled directly in the higher-level 
course, ACT Test score was a statistically significant predictor of degree completion for four of 
the six course pairs. The exceptions were students who enrolled directly in Elementary Algebra 
(and did not take Arithmetic) and Intermediate Algebra (and did not take Elementary Algebra). 
This finding may be attributed to the restriction in the range of ACT Mathematics scores for 
these courses (functional range of a maximum of 5 score points) in conjunction with small 
sample sizes and the smaller proportions of students in these courses who completed degrees. 

Among the English, Elementary Algebra/Intermediate Algebra, and Intermediate 
Algebra/College Algebra course pairs, taking the lower-level course was associated with a higher 
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probability than expected of completing a Bachelor’s degree in six years, had students enrolled 
directly in the higher-level course. In general, this result occurred at all ACT Test score levels. 
Similar results occurred for the Intermediate Algebra/College Algebra course pair for three-year 
Associate’s degree completion. Figures 19 and 20 show the results for Developmental/Standard 
English Composition and Intermediate Algebra/College Algebra, respectively. 



ACT English score 

Figure 19. Probability of completing an Associate’s degree in three years or a Bachelor’s degree 
in five or six years for students who took Standard English Composition with and without taking 
Developmental English Composition. 
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Figure 20. Probability of completing an Associate’s degree in three years or a Bachelor’s degree 
in five or six years for students who took College Algebra with and without taking Intermediate 
Algebra. 


There was little or no benefit with respect to five-year Bachelor’s degree completion. The 
reason for this result, of course, is that institutions typically do not award credit towards earning 
a degree for developmental coursework. 

We also compared the probability curves for five-year Bachelor’s degree completion of 
students who enrolled directly into the higher-level course with those for six-year degree 
completion of students who first took the lower-level course. In a pattern similar to the 
descriptive results discussed previously, students who took the lower-level courses had a higher 
probability of completing a six-year Bachelor’s degree than similar students who took the 
higher-level course had in completing either a five- or six-year Bachelor’s degree. 
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Lower-level course grades as predictors of GPA at graduation/last term and degree 
completion for full-time students. ACT Test score was not a statistically significant predictor of 
degree completion in the joint models that included lower-level course grade. 16 As noted earlier, 
this result is what one should expect: Placement decisions for most students in the study were 
based on ACT Test scores, resulting in restriction in the range of ACT test scores for students in 
the lower-level courses. Moreover, if lower-level coursework is effective in providing the 
knowledge and skills that are necessary for success in the next course, then test scores obtained 
before taking lower-level coursework no longer reflect what students know and are able to do, 
once they take the coursework. 

When predicting cumulative GPA at graduation/last term, only a grade of A in 
developmental English or mathematics courses was associated with a higher probability of 
success than would be expected, had students enrolled directly in the corresponding higher-level 
course. Figure 21 illustrates this result for Developmental English Composition/Standard 
English Composition. 


16 The sole exception was Associate’s degree completion for students who took Arithmetic prior to enrolling in 
Elementary Algebra. 
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ACT English score 

Figure 21. Probability of a cumulative GPA at graduation/last term of 3.0 or higher for students 
who took Standard English Composition, by Developmental English Composition grade. 


This result was true for all GPA levels and all lower-level English and mathematics 
courses except Arithmetic. In Arithmetic, a grade of B was also associated with a higher 
probability of a 3.0 or higher GPA, but only for students with higher ACT Test scores (see 
Figure 22). Arithmetic course grade was not a statistically significant predictor of 2.0 or higher, 
or 2.5 or higher, cumulative GPA at graduation/last term. 
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ACT Mathematics score 

Figure 22. Probability of a cumulative GPA at graduation/last term of 3.0 or higher for students 
who took Elementary Algebra, by Arithmetic grade. 


For Developmental Reading (before American History), a grade of A was associated with 
a slightly higher probability (maximum difference of .05; see Table C-5) of achieving a 
cumulative GPA at graduation/last term of 2.0 or higher but only for those students with very 
low ACT Reading scores. Figure 23 illustrates this result for the 2.0 or higher GPA outcome. 
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ACT Reading score 

Figure 23. Probability of a cumulative GPA at graduation/last term of 2.0 or higher for students 
who took American History, by Developmental Reading grade. 

Lower-level course grade was a significant predictor for three-year Associate’s and five- 
year Bachelor’s degree completion for almost all course pairs (except Arithmetic/Elementary 
Algebra). However, only a lower-level course grade of A was associated with higher 
probabilities of achieving a three-year Associate’s degree for students enrolling in 
Developmental English Composition or Intermediate Algebra. The result for the Intermediate 
Algebra/College Algebra course pair is illustrated in Figure 24. Similarly, a grade of A in 
Developmental English Composition or Intermediate Algebra was associated with higher 
probabilities of completing a five-year Bachelor's degree. A grade of A in Developmental 
English Composition or Intermediate Algebra increased the probability of completing a 
Bachelor’s degree within five years by .05 or .06, respectively, for students with very low ACT 


Test scores. 
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Figure 24. Probability of completing an Associate’s degree in 3 years for students who took 
College Algebra, by Intermediate Algebra grade. 


Lower-level course grade was not a statistically significant predictor of Bachelor’s degree 
completion in six-years, or could not be modeled, for almost all course pairs. The one exception 
was Elementary Algebra/Intermediate Algebra, where an A grade was associated with a .14 
increase in the probability of completing a Bachelor’s degree in six years, regardless of ACT 
Mathematics score. 

In contrast, the probability of completing an Associate’s degree in three years or of 
completing a Bachelor’s degree in five years associated with any grade in Developmental 
Reading was similar to or lower than that of students who enroll directly in Psychology (see 
Figure 25) or in American History. 
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ACT Reading score 

Figure 25. Probability of completing an Associate’s degree in 3 years for students who took 
Psychology, by Developmental Reading grade. 

Full-time/part-time enrollment status. For the vast majority of course pairs, student 
groups, and outcome variables, students’ first-year enrollment status was a statistically 
significant predictor of college success. Full-time students were more likely than part-time 
students to achieve a C/2.0 or higher, or B/3.0 or higher, outcome. The exceptions were first- 
time course outcomes for Elementary Algebra and Developmental English Composition, and 
five-year Bachelor’s degree completion for Developmental English Composition, Arithmetic, 
and Elementary Algebra. The results are shown in Appendix D. 

Tables D-l through D-6 summarize the estimated probabilities of success for full- and 
part-time students who did and did not take the lower-level course prior to taking the higher- 
level course. The other predictor variables (ACT Test score and grade in lower-level course) 
were held constant at their respective means. Of students who first enroll in the lower-level 
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course, the probabilities of subsequent college success for full-time students exceeded those of 
part-time students by .08 to .13, on average. Of students who enrolled directly in a higher-level 
course, the differences in probability of subsequent college success between full- and part-time 
students were larger, ranging from .14 to .32, on average. 

Success in the higher-level course (B or higher and C or higher outcomes) of each course 
pair depended on student enrollment status, as well as on whether they first took the lower-level 
course. Full-time students who enrolled directly in a higher-level course had higher estimated 
probabilities of a B or higher or C or higher grade than similar students who first took the 
associated lower-level course . We found similar results for part-time students in 
Developmental Reading/American History and in Developmental Reading/Psychology. 
However, part-time status was associated with higher estimated probabilities of a C or higher 
grade for all other course pairs, if students first enrolled in the associated lower-level course. 

When examined within enrollment status group, probabilities of success associated with 
taking and not taking the lower-level course before the higher-level course differed substantially 
for early college outcomes: For part-time students, probabilities of success associated with Term 
1 (2.0 and 3.0 or higher), first-year (2.0 or higher), and second-year (2.0 or higher) 
GP A/persistence indicators were much higher for students who first enroll in the lower-level 
course than for those who enroll directly in the higher-level course, regardless of the course pair 
examined. The differences were particularly large for Term 1 outcomes. In comparison, the 
differences in probabilities of early college success associated with taking and not taking the 
lower-level course for full-time students were considerably smaller. Though differences in 
probability of success for Term 1 generally favored full-time students first taking the lower-level 


17 An exception was Arithmetic/Elementary Algebra. 
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course, these differences decreased over time and frequently reversed themselves to favor 
students who first enroll in the higher-level course. 

Estimated probabilities of achieving a GPA of 2.0, 2.5, or 3.0 at graduation/last term 
enrolled generally differed within enrollment status group by whether students first enroll in the 
lower-level course. Probabilities did not differ substantially for part-time students who did and 
did not first enroll in the lower-level course; we did find differences for full-time students, 
however. In general, these differences favored full-time students who enroll directly into the 
higher-level course. Conversely, for Arithmetic/Elementary Algebra, full-time students who first 
enroll in the developmental course had a somewhat higher probability of a 2.0 or higher, or 2.5 
or higher, GPA at graduation than their counterparts who enroll directly in the higher-level 
course. 

Timely degree completion universally favored full-time students, but also depended on 
whether students took a lower-level course prior to taking the higher-level course. Part-time 
students who did and did not take Developmental Reading prior to American History or 
Psychology were similarly likely to complete an Associate’s or Bachelor’s degree (five or six 
years) in a timely way. For full-time students, enrolling directly in Standard English 
Composition, American History, or Psychology was associated with higher probabilities of 
completing an Associate’s or a Bachelor’s degree (five or six years) than for those first enrolling 
in the corresponding lower-level course. In contrast, full-time students who first enroll in 
Elementary Algebra had a higher probability of completing a Bachelor’s degree in six years than 
students who enroll directly into Intermediate Algebra. 

Part-time students taking Intermediate Algebra and College Algebra had higher estimated 
probabilities of timely Associate’s and six-year Bachelor’s degree completion than those taking 
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only College Algebra. Similar finding occurred for part-time students taking Developmental 
English Composition before Standard English Composition or Elementary Algebra before 
Intermediate Algebra, but only for six-year degree completion. 

College Type. As shown in the tables in Appendix B, college type was a statistically 
significant predictor of some outcomes, although not as frequently as full-time/part-time 
enrollment status. There were no simple consistent patterns where college type was statistically 
significant. For example, for the English Composition course pair, students first enrolling at two- 
year colleges had higher probabilities of Term 1 and Year 1 success (Term 1 2.0 or higher and 
3.0 or higher, Year 1 3.0 or higher) than those enrolling at four-year institutions. This finding 
occurred regardless of whether students first took the lower-level course before taking the 
higher-level course. 

In contrast, students at four-year institutions had higher estimated probabilities of success 
related to Year 2 outcomes (2.0 or higher only) than did students at two-year colleges for all 
course pairs except Arithmetic/Elementary Algebra. These results were paralleled at the 3.0 
level, but only for students who enrolled directly in higher-level courses, and were not found for 
Elementary Algebra/Intermediate Algebra. 

Not surprisingly, with the exception of students who took Arithmetic or Developmental 
Reading prior to American History, students who first enroll at a four-year institution had higher 
estimated probabilities of completing a Bachelor’s degree within five years than students who 
first enroll at a two-year institution. 

Discussion 

Previous research has shown that developmental students are less successful overall than 
students who do not take developmental courses (Attewell et al., 2006; NCES, 2004). In this 
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study, we examined the effectiveness of developmental courses from another, more basic 
perspective: Do students derive any benefit at all from taking developmental courses? In other 
words, even if students who take developmental courses are less successful later on than non- 
developmental students, are the developmental students more successful than they would have 
been if they had not taken the developmental courses? To answer this question, we compared 
the college success of developmental and non-developmental students who had the same ACT 
Test scores, enrollment status, and who enrolled in similar institutions. We measured college 
success using many different outcome variables: completing the subsequent course with a 
satisfactory grade; cumulative GPA/persistence the first term, first year, and second year; 
cumulative GPA at graduation; and degree completion (Associate’s in three years or Bachelor’s 
in five or six years). We compared the conditional probabilities of success, given ACT Test 
score and enrollment status, of six groups of students who took particular developmental courses 
with the corresponding conditional probabilities of students who enrolled directly in the 
associated higher-level course. 

The overall results reported here confirm previous research findings: Taken as a whole, 
developmental students appear less successful overall than non-developmental students in terms 
of GPA/persistence over time and degree completion within specified periods of time. Our 
results also show, however, that particular subgroups of developmental students do benefit, 
especially when we take into account the greater time they need to complete their degrees. 
Success in College Depends on Prior Academic Preparation 

It bears repeating that better prepared students (as measured by their ACT Test scores) 
are more successful in college than less prepared students, no matter what outcome, short-term or 
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long-term, that we consider. This result pertains both to students who take developmental 
courses and to students who enroll directly in higher-level courses. 

Among students who take developmental courses, however, the grade they receive in 
these courses is often a better predictor of long-term academic success than their ACT Test 
scores. This finding is not surprising. If developmental coursework is effective in providing the 
knowledge and skills that are necessary for success in the next course, then test scores obtained 
before taking developmental coursework no longer reflect what students know and are able to do 
after they take the course. Moreover, placement decisions for the vast majority of students in the 
study were based on ACT Test scores. The resulting distributions of ACT test scores for the 
lower-level courses were restricted, in some cases quite severely. The range of scores for 
Arithmetic, Elementary Algebra, and Intermediate Algebra each included only five ACT Test 
score points. 

This finding also underscores the importance, when validating test scores or other 
measures for course placement, of ensuring that no intervening instruction has occurred (or else 
is statistically controlled for). ACT's Course Placement Service (ACT, 2012b) recommends that 
institutions include in their validity studies data only from first-time students without prior 
developmental instruction. The ACT Course Placement Service also recommends that 
institutions identify students who are taking developmental coursework at the same time as 
standard college-level courses. 

Simply Taking Developmental Courses Results in Few Long-Term Benefits 

For most of the developmental courses, simply taking the courses (without considering 
the grades earned in them) did not result in any apparent benefit to success in the subsequent 
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higher-level courses. The two exceptions were Arithmetic and Intermediate Algebra, where 
there was modest benefit (maximum increases in probability of .09 and .05, respectively). 

Full-time students who took a developmental course, as a group, however, did improve 
their chances of successful Term 1 GP A/persistence to Term 2 and (to a lesser extent) Year 1 
GP A/persistence to Year 2 outcomes. This result could be attributable to how institutions treat 
developmental course credit and grades. Most colleges offer institutional credit for 
developmental courses, but the credit does not typically count toward a degree (Parsad & Lewis, 
2003), and the corresponding developmental course grades might or might not be used in 
calculating cumulative GPA. Follow-up with the two states that provided the large majority of 
the college outcome data for this study revealed that for one state, developmental grades were 
included in cumulative GPA calculations. For the other state, the inclusion/exclusion decision 
was institution-specific and not determined at the state level. The very high percentages of 
students receiving C or higher or passing grades in these courses, and the relatively high Term 1 
and Year 1 cumulative GPAs, are therefore not surprising. 

After the first two years, benefits associated with developmental coursework tended to 
decline and, in some cases, disappear. Taking developmental courses did improved full-time 
students’ chances of achieving a 2.0, 2.5, or 3.0 cumulative GPA at graduation for students who 
took Arithmetic. There was also a benefit with respect to completing an Associate’s degree 
within three years (Intermediate Algebra/College Algebra) or completing a Bachelor’s degree in 
five or six years (Developmental English Composition and mathematics courses). To some 
extent these findings are consistent with those of other studies that looked at long-term college 
outcomes (e.g., Adelman, 1999; Calcagno & Long, 2008), which found that taking 
developmental coursework did not improve later college success. These studies used different 
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methodological approaches and college outcomes than those used here, however. The findings 
here illustrate the importance of taking time to degree into consideration in conducting such 
research, with consideration for the delayed accumulation of credit hours resulting from taking 
developmental coursework. 

Across all subject areas and outcomes studied, Developmental Reading appeared to be 
least beneficial for improving the academic preparedness of entering students. For most of the 
institutions in this study, Developmental Reading could be taken prior to, concurrent with, or 
following college-level social science courses. Students who took Developmental Reading after 
taking college-level social science courses were excluded from the analyses, thereby limiting the 
numbers of students who actually took Developmental Reading. It is unclear the extent to which 
having all students take Developmental Reading prior to or concurrent with standard college- 
level social sciences coursework would change or improve these results. 

The potential benefits of taking developmental coursework was also found to depend on 
other factors: academic preparedness, as measured by ACT Test scores; the course grade in the 
lower-level (developmental) course; and first-year enrollment status. The following sections 
address each of these factors. 

The Benefit of Taking Developmental Courses Mostly Depends on the Grades Earned in 
Them 

Consistent with findings by Perkhounkova, Noble, and Sawyer (2005) and others 
(Bettinger & Long, 2005a; Boatman & Long, 2010; Calcagno & Long, 2008), the benefits of 
taking developmental coursework depend on the grade in the developmental course. Pass/fail 
grades were found to be of very limited value as indicators of what students learned in the 
developmental course, and of students’ likely success later in college. This finding may be 
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attributable to the very high percentages of students receiving passing grades, which would limit 
the extent to which students could be differentiated in terms of what they know and are able to 
do. 

In contrast, A grades in the developmental course were associated with higher 
probabilities of success than expected, had students enrolled directly in the higher-level course. 
This finding was consistent across course pairs. We obtained a similar result for some (but not 
all) of the course pairs for students who earned a B or higher grade in the developmental course. 
The higher probabilities associated with B grades depended on ACT Test score: The benefit 
associated with B grades typically occurred for students with very low ACT Test scores. 

The benefits associated with receiving an A or B grade in the developmental courses 
tended to decrease over time, paralleling the general results described earlier. For later college 
success outcomes, only a grade of A in the developmental course was associated with a higher 
probability of success than would have been expected (and then, only for a few outcomes). 

The Benefit of Developmental Courses Depends on First-Year Enrollment Status 

The results of this study show that full-time students are more likely than part-time 
students to succeed in college, regardless of the outcome being considered. Part-time students, 
however, appeared to derive more benefit from taking developmental courses than full-time 
students did. For example, part-time students benefited from taking Developmental English 
Composition, Elementary Algebra, and Intermediate Algebra in terms of a C or higher grade in 
the higher-level course. Moreover, part-time students who took a developmental course were 
more likely to be successful in Term 1, Year 1, and Year 2 than students with similar ACT Test 
scores who did not take these courses and who enrolled directly in a higher-level course. Part- 
time students who took Developmental English Composition, Elementary Algebra, or 
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Intermediate Algebra also had better chances of completing a three-year Associate’s 
(Intermediate Algebra only) or six-year Bachelor’s degree than would be expected of non- 
developmental students with similar ACT Test scores. 

The results for full-time students, in general, did not parallel these findings. For most 
outcomes for full-time students, taking the developmental course did not appear to increase their 
long-term chances of success. 

The associations between first-year enrollment status and the effectiveness of the 
developmental courses were interesting and unexpected. Most of the recent research on the 
effectiveness of developmental instruction has focused on two-year or four-year college students 
(but not on both), or on degree-seeking students only, or on full-time students only. Given the 
current trend for nontraditional (e.g., adult) students to reenroll part-time in two-year colleges, 
either to refresh their skills or obtain new ones, not considering part-time students when 
evaluating the effectiveness of developmental instruction leaves a gap in our knowledge. 

The Practical Benefits of Developmental Coursework 

An important consideration is the extent to which the benefits of developmental 
instruction are of practical value. For example, in one scenario students with lower ACT 
Mathematics scores who take Arithmetic before taking Elementary Algebra increase their 
chances of success in Elementary Algebra over what might be expected, had they enrolled 
directly into Elementary Algebra. However, even with this increase in probability of success, 
Arithmetic students still have about a 50/50 or smaller chance of earning a C or higher in 
Elementary Algebra (see Figure 9). In another scenario, students with very low ACT Reading 
scores benefit in Psychology from taking Developmental Reading if they get a grade of A in the 
course, but have greater than a 6 in 10 chance of succeeding in Psychology without taking the 
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developmental course. For either scenario, is providing developmental coursework of sufficient 
benefit to justify the costs of providing it? When evaluating the effectiveness of developmental 
instruction, researchers and policymakers need to consider both the baseline success rate for the 
higher-level course and the expected success rate, given developmental instruction. 

The Bottom Line: Does Developmental Coursework Benefit Students? 

Both two- and four-year colleges are under pressure to increase degree completion rates 
and to decrease developmental education on the grounds that it does not benefit students 
(Bettinger & Long, 2005a; Gonzales, 2012; Merisotis & Phipps, 2000; “Experts: Remedial 
classes need fixing,” 2012; Saxon & Boylan, 2001). The basis for these findings is that students 
who take developmental courses are not as successful in college as students who do not need to 
take developmental courses. Our research confirms this finding: Developmental students as a 
group were not as successful in college as non-developmental students as a group, with respect to 
GP A/persistence over time and degree completion within a fixed time period. However, 
consideration of the additional time required to complete a bachelor’s degree by developmental 
students showed that these students can complete bachelor’s degrees in six years at a rate similar 
to or higher than that of non-developmental students in five years. 

Our primary goal, however, was to investigate benefit from another perspective: Do 
students derive any benefit from taking developmental courses, in the sense that they are more 
successful than similar students who do not take developmental courses? We defined similarity 
in terms of students’ readiness for college-level work, as measured by their ACT Test scores, by 
their enrollment status, and by the type of college in which they enrolled. We compared the 
conditional probability of success, given ACT Test score and enrollment status, of groups of 
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students who took particular developmental courses with those who took the next higher-level 
course. 

For six-year bachelor’s degree completion, the benefit was large enough to match the 
success of students enrolling in higher-level courses. For the other outcomes, students did 
benefit, but typically only if they earned an A in the developmental course. For some course 
pairs, students who entered the developmental course with low ACT Test scores and who earned 
a B in the course also derived benefit. There are a variety of explanations for these findings, only 
some of which could be explored here. Other considerations include the following: 

• Noncognitive characteristics of students . Although prior academic achievement is a 
strong predictor of success in college, noncognitive characteristics are also important. 
Examples of such characteristics include psychosocial characteristics and behavior 
(principally motivation and academic discipline; see Allen & Robbins, 2010 and Allen, 
Robbins, & Sawyer, 2010), family environment (support and encouragement to succeed 
in college), and life situations (e.g., care for dependents, the need to work while in 
college). Noncognitive characteristics affect grades earned in high school as well as in 
college (Goldman & Hewitt, 1975; Goldman, Schmidt, Hewitt, & Fisher, 1974; Goldman 
& Widawski, 1976; Stiggins, Frisbie, & Griswold, 1989). Background characteristics 
(e.g., gender, race/ethnicity, family income) are also related to noncognitive 
characteristics (Allen & Robbins, 2010; Angrist, Lang, & Oreopoulos, 2009; Engle & 
Tinto, 2008; Hurtado, Laird, & Perorazio, 2010; Le, Casillas, Robbins, & Langley, 2005; 
Lotkowski, Robbins, & Noeth, 2004; Tym, McMillion, Barone, & Webster, 2004; 
Young, 2001; Zwick & Sklar, 2005). 
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Although the data for this study did not include measures of noncognitive characteristics, 
we suspect that they explain, to a large extent, developmental students’ lower overall 
levels of success: Developmental students might be disadvantaged relative to non- 
developmental students in these noncognitive characteristics, and are therefore less 
successful in college. Noncognitive characteristics could also explain, in part, 
developmental students’ lower test scores to begin with. Research with ACT’s 
ENGAGE® for college students has shown the relationships between students’ 
noncognitive characteristics and college retention, GPA, and timely degree attainment 
(e.g., Allen & Robbins, 2010; Allen, Robbins, & Sawyer, 2010). The research has also 
shown that academic discipline predicts success in Elementary Algebra (Robbins, Allen, 
Casillas, Peterson, & Le, 2006) and that student behaviors during the semester 
(participation in group work and lecture, attendance, and homework completion) predict 
end-of-semester knowledge (as measured by ACT COMPASS) and course success (Li, 
Zelenka, Buonaguidi, Beckman, Casillas, Crouse, Allen, Hanson, Acton, & Robbins, 
2012 ). 

• The degree to which developmental education extends beyond providing developmental 
instruction . As noted by Boylan (1995), developmental education includes providing 
developmental courses, but also includes advising/counseling and other services that 
address needs related to students’ noncognitive characteristics. Support programs 
provide academic support for academically underprepared students, and social supports 
to encourage social integration at the institution (Padgett & Keup, 2011). They may 
include freshman orientation, first year seminars, summer bridge programs, mentoring, 
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advising, and counseling for selected population subgroups, course placement, and 
learning communities (Muraskin, 1997). 

• Sustained support systems throughout college . Relatively large benefits of 
developmental instruction were observed in the first year of college, but declined 
substantively in subsequent years. One could speculate that the apparent decline in 
benefits from developmental instruction after the first year could be due, in part, to the 
lack of support systems after the first or second year. Support programs for at-risk 
students are typically introduced early in college (Tinto, 2004). They are also 
widespread; for example, in 2011 researchers from the National Resource Center for the 
First-Year Experience and Students in Transition reported that 87% of responding U. S. 
postsecondary institutions (N = 1,019) offered a first-year seminar. Of these institutions, 
over half had nearly all first-year students participating in the program (Padgett & Keup, 
2011). In contrast, only about one-third of responding institutions had initiatives for 
sophomores; these initiatives typically emphasized retention, satisfaction, and student 
engagement (Keup, Gahagan, & Goodwin, 2010). In comparison, senior-year programs 
receive little attention in the literature, and studies that do exist focus on senior 
“capstone” experiences (Padgett & Kilgo, 2012), with little commonality in the definition 
of what “capstone” means (Brownell & Swaner, 2010). 

• The structure and content of developmental courses . The benefits of developmental 
coursework also depend on the extent to which the course provides students with the 
skills and knowledge students need to be successful in higher-level courses. To the 
extent that course content is not aligned with that of higher-level courses, students are 
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less able to acquire the knowledge and skills they need to be successful in the higher- 
level course. 

• The structure of course placement systems . The accuracy and benefits resulting from 
course placement systems rest on the measures used to make course placement decisions, 
the cutoff values used on those measures, and any rules established concerning their use 
(e.g., mandatory vs. voluntary course placement, students being allowed to take the 
developmental courses only before or concurrently with the associated higher-level 
course, etc.). For example, several students were excluded from the Developmental 
Reading/American History and Developmental Reading/Psychology analyses because 
they took the lower-level course after the higher-level course. It may be that these 
students would have done better in the higher-level course, had they taken the lower-level 
course before or concurrently with the higher-level course. 

• The cost and fatigue factors associated with taking full-term developmental courses 
(possibly for several terms) . Taking developmental courses may be too expensive, too 
tiring, or too frustrating for many students; they simply wear out and give up. In 
response, some companies have started offering targeted brush-up instruction delivered 
on-line (American Education Corporation, 2009; Blackboard, Inc., 2012; Pearson 
Education Inc., 2012; PLATO Learning, Inc., 2012; see also Tong, Saxon, Boylan, 
Bonham, & Smith (2012) for a detailed summary of developmental mathematics 
software). The software administers on-line diagnostic tests, provides instruction in the 
areas where deficiencies are noted, and administers a mastery test to document 
acquisition of the targeted knowledge and skills. This method for providing 
developmental instruction has obvious advantages in time, cost, and convenience to 
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students, but its effectiveness, like that of traditional developmental courses, needs to be 
studied. 

We can only speculate on the extent to which these considerations influence the benefits 
of developmental programs. Research to confirm or refute their influences would need to 
incorporate data on students’ noncognitive characteristics, their developmental coursework, as 
well as detailed information on the treatments and interventions that they participated in 
throughout college. 
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Appendix A 


Pooled Descriptive Results 

Tables A-l through A- 6 


Note: All results pertaining to six-year degree completion were based on only 22 of the 35 four-year institutions 
the study. 
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Summary of Student Groups for Elementary and Intermediate Algebra 
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Summary of Student Groups for Intermediate and College Algebra 
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Appendix B 

Hierarchical Logistic Regression Models for Predicting Success in College 

Tables B1 through B 13 


Note: All results pertaining to six-year degree completion were based on only 22 of the 35 four-year institutions in 
the study. 
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Table B-3 


Variance Components of Hierarchical Logistic Regression Models for Predicting Subsequent 
Academic Success after Taking Developmental English Composition and/or Standard English 
Composition 


Outcome variable 



ACT Test score 




Type 

Level 

Intercept 

slope 

All students who took Developmental English Composition before Standard English Composition 

Success in Std. English Comp 

C or higher 
B or higher 

0.08929 

0.17946 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.17750 

0.17946 

— 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.06308 

0.10110 

0.00263 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.05090 

0.07516 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.27496 

0.23989 

0.14624 

— 

Associate’s degree within 3 years 


0.35980 

— 

Bachelor’s degree within 5 years 


0.11281 

— 

Bachelor’s degree within 6 years 


— 

— 


Students who took Developmental English Composition (grade scale A-F) before Standard English 
Composition 


Success in Std. English Comp 

C or higher 
B or higher 

0.08636 

0.15723 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.29191 

0.20817 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.11910 

0.12688 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.10767 

0.08441 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.19937 

0.18813 

— 

Associate’s degree within 3 years 


0.41048 

— 

Bachelor’s degree within 5 years 


— 

— 

Bachelor’s degree within 6 years 


— 

— 


(continued on next page) 
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Table B-3 (continued) 


Outcome variable 


Im. 


Level 


ACT Test score 
Intercept slope 


Students who took Developmental English Composition (grade scale pass/fail) before 
Standard English Composition 


Success in Std. English Comp 

C or higher 
B or higher 

— 

— 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.14632 

0.09210 

— 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

— 

— 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.12661 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.58299 

0.50159 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


— 

— 

Students who enrolled directly in 

Standard English Composition 


Success in Std. English Comp 

C or higher 
B or higher 

0.15645 

0.15480 

0.00056 

0.00065 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.61474 

0.17222 

0.00039 

0.00047 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.10664 

0.07340 

0.00022 

0.00080 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.10692 

0.07767 

0.00085 

0.00194 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.25533 

0.23284 

0.23373 

0.00045 

0.00047 

0.00052 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.15930 

0.12857 

0.13163 

0.00067 

0.00049 


Note: Statistically significant (p < .01) variance components are listed. Non-statistically significant variance 

components are noted in the table as ’ and were not included in the final models. Variance components for models 
that could not be developed are left blank. 



Fixed Effects of Hierarchical Logistic Regression Models for Predicting Subsequent College Success after Taking Arithmetic and/or 
Elementary Algebra 
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Table B-5 

Variance Components of Hierarchical Logistic Regression Models for Predicting Subsequent 
Academic Success after Taking Arithmetic and/or Elementary Algebra 


Outcome variable 



ACT Test score 
slope 

Jype 

Level 

Intercept 

All students who took Arithmetic before Elementary Algebra 

Success in Elementary Algebra 

C or higher 
B or higher 

0.49421 

0.35677 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0. 17576 
0.09754 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0. 15184 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

— 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

— 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


— 

— 

Students who took Arithmetic (grade scale A-F) before Elementary Algebra 

Success in Elementary Algebra 

C or higher 
B or higher 

0. 12691 
0. 27294 

— 

1 st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0. 10343 

— 

1 st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

— 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

— 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

— 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0. 22884 

— 


(continued on next page) 
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Table B-5 (continued) 


Outcome variable 


ACT Test score 
Intercept slope 

Type 

Level 

Students who took Arithmetic (grade scale pass/fail) before Elementary Algebra 

Success in Elementary Algebra 

C or higher 
B or higher 


1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 


Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 



Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


Students who enrolled directly in Elementary Algebra 


Success in Elementary Algebra 

C or higher 
B or higher 

2.18399 

1.67714 

0.01145 

1st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.48859 

0.35618 

— 

1st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.36947 

0.39166 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.23418 

0.27330 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.38943 

0.33374 

0.26062 

— 

Associate’s degree within 3 years 


0.40382 

— 

Bachelor’s degree within 5 years 


0.20929 

— 

Bachelor’s degree within 6 years 


— 

— 


Note: Statistically significant (p < .01) variance components are listed. Non-statistically significant variance 

components are noted in the table as ’ and were not included in the final models. Variance components for models 
that could not be developed are left blank. 


Fixed Effects of Hierarchical Logistic Regression Models for Predicting Subsequent College Success after Taking Elementary and/or 
Intermediate Algebra 
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Table B-7 

Variance Components of Hierarchical Logistic Regression Models for Predicting Subsequent 
Academic Success after Taking Elementary and/or Intermediate Algebra 


Outcome variable 

Intercept 

ACT Test score 
slope 

Jype 

Level 

All students who took Elementary Algebra before Intermediate Algebra 

Success in Intermediate Algebra 

C or higher 
B or higher 

0.66402 

0.45276 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.27417 

0.18496 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.15100 

0.22901 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.06322 

0.17924 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.16752 

0.15910 

— 

Associate’s degree within 3 years 


— 

— 

Bachelor’s degree within 5 years 


— 

— 

Bachelor’s degree within 6 years 


— 

— 


Students who took Elementary Algebra (grade scale A-F) before Intermediate Algebra 


Success in Intermediate Algebra 

C or higher 
B or higher 

0.09910 

0.16935 

1 st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.26032 

0.16679 

1 st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.11219 

0.14733 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.03658 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.19697 

0.16410 

0.14685 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


— 


(continued on next page) 
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Table B-7 (continued) 


Outcome variable 



ACT Test score 
slope 

Type 

Level 

Intercept 

Students who took Elementary Algebra (grade scale pass/fail) before Intermediate Algebra 

Success in Intermediate Algebra 

C or higher 
B or higher 



1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.28277 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

— 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 



Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 




Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


Students who enrolled directly in Intermediate Algebra 


Success in Intermediate Algebra 

c 

C or higher 
B or higher 

1.32347 

1.09427 

— 

1st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.23359 

0.21292 

0.00643 

1st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.16626 

0.17327 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.12994 

0.06741 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.34037 

0.32218 

0.29119 

— 

Associate’s degree within 3 years 


0.26264 

— 

Bachelor’s degree within 5 years 


0.12104 

— 

Bachelor’s degree within 6 years 


0.62077 

— 


Note: Statistically significant (p < .01) variance components are listed. Non-statistically significant variance 

components are noted in the table as ’ and were not included in the final models. Variance components for models 
that could not be developed are left blank. 
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Table B-9 


Variance Components of Hierarchical Logistic Regression Models for Predicting Subsequent 
Academic Success after Taking Intermediate and/or College Algebra 


Outcome variable 

Intercept 

ACT Test score 
slope 

Jype 

Level 

All students who took Intermediate Algebra before College Algebra 

Success in College Algebra 

C or higher 
B or higher 

0.19164 

0.27598 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.37449 

0.27595 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.09423 

0.16734 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.07159 

0.07084 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.36044 

0.34729 

0.37808 

— 

Associate’s degree within 3 years 


0.16667 

— 

Bachelor’s degree within 5 years 


0.13578 

— 

Bachelor’s degree within 6 years 


0.22006 

— 


Students who took Intermediate Algebra Algebra (grade scale A-F) before College Algebra 


Success in College Algebra 

C or higher 
B or higher 

0.12159 

0.20628 

— 

1 st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.32706 

0.18251 

— 

1 st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.12773 

0.09869 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.07690 

0.07878 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.33498 

0.31125 

0.38553 

— 

Associate’s degree within 3 years 


— 

— 

Bachelor’s degree within 5 years 


— 

— 

Bachelor’s degree within 6 years 


— 

— 


(continued on next page) 








115 


Table B-9 (continued) 


Outcome variable 



ACT Test score 
slope 

Type 

Level 

Intercept 

Students who took Intermediate Algebra (grade scale pass/fail) before College Algebra 

Success in College Algebra 

C or higher 
B or higher 

0.40227 

0.51184 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.23230 

0.24585 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.07408 

0.18071 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.23449 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.19205 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


— 

— 

Students who enrolled directly in College Algebra 

Success in College Algebra 

C or higher 
B or higher 

0.15507 

0.17354 

0.00099 

1st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.51557 

0.19374 

0.00155 

1st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.05665 

0.04190 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.07098 

0.03416 

0.00205 

0.00347 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.22364 

0.20803 

0.21877 

0.00105 

0.00114 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.20977 

0.07946 

0.05865 

— 


Note: Statistically significant (p < .01) variance components are listed. Non- statistically significant variance 

components are noted in the table as and were not included in the final models. Variance components for models 
that could not be developed are left blank. 
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Table B-ll 

Variance Components of Hierarchical Logistic Regression Models for Predicting Subsequent 
Academic Success after Taking Developmental Reading and/or American History 


Outcome variable 


ACT Test score 
Intercept slope 

Jype 

Level 

All students who took Developmental Reading before American History 

Success in American History 

C or higher 
B or higher 

0.31065 

0.30331 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.35721 

0.28639 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.09287 

0.22655 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

— 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.28860 

0.24864 

0.25181 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.33096 

0.26682 

— 

Students who took Developmental Reading (grade scale A-F) 1 

before American History 


Success in American History 

C or higher 
B or higher 

0.26340 

0.27164 

— 

1 st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.54123 

0.34897 

— 

1 st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.14352 

0.27511 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

— 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.20955 

0.22620 

0.16866 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.39419 

— 


(continued on next page) 
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Table B-ll (continued) 


Outcome variable 



ACT Test score 
slope 

Type 

Level 

Intercept 

Students who took Developmental Reading (grade scale pass/fail) before American History 

Success in American History 

C or higher 
B or higher 

0.14183 

0.19479 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.14484 

0.19109 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.34642 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

— 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

— 

— 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


— 

— 

Students who enrolled directly in American History 

Success in American History 

C or higher 
B or higher 

0.12638 

0.15353 

0.00124 

0.00097 

1st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.88759 

0.22461 

0.00033 

0.00056 

1st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.27408 

0.07790 

0.00037 

0.00062 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.12855 

0.05523 

0.00063 

0.00140 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.27720 

0.24357 

0.22552 

0.00035 

0.00035 

0.00048 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.25289 

0.11045 

0.10999 

0.00024 


Note: Statistically significant (p < .01) variance components are listed. Non- statistically significant variance 

components are noted in the table as and were not included in the final models. Variance components for models 
that could not be developed are left blank. 
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Table B-13 

Variance Components of Hierarchical Logistic Regression Models for Predicting 
Subsequent Academic Success after Taking Developmental Reading and/or Psychology 


Outcome variable 



ACT Test score 




Jype 

Level 

Intercept 

slope 

All students who took Developmental Reading before Psychology 

Success in Psychology 

C or higher 
B or higher 

0.24388 

0.47088 

0.00363 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.44383 

0.30596 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.04918 

0.16624 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.06595 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.18723 

0.15626 

0.13265 

— 

Associate’s degree within 3 years 


0.28092 

— 

Bachelor’s degree within 5 years 


— 

— 

Bachelor’s degree within 6 years 


— 

— 

Students who took Developmental Reading (grade scale A-F) before Psychology 

Success in Psychology 

C or higher 
B or higher 

0.20358 

0.46950 

— 

1 st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.68297 

0.41632 

— 

1 st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.06196 

0.24331 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.19738 

— 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.17500 

— 

Associate’s degree within 3 years 


0.39011 

— 

Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


— 

— 


(continued on next page) 
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Table B-13 (continued) 


Outcome variable 



ACT Test score 
slope 

Type 

Level 

Intercept 

Students who took Developmental Reading (grade scale pass/fail) before Psychology 

Success in Psychology 

C or higher 
B or higher 

0.62343 

0.62160 

— 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.21356 

0.13496 

— 

1 st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

— 

— 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 



Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.12779 


Associate’s degree within 3 years 


— 

— 


Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


Students who enrolled directly in Psychology 


Success in Psychology 

/ OJ 

C or higher 
B or higher 

0.14657 

0.20649 

0.00038 

0.00052 

1st term GPA/persist to term 2 

2.0 or higher 

3.0 or higher 

0.97216 

0.21932 

0.00029 

1st year GPA/persist to year 2 

2.0 or higher 

3.0 or higher 

0.16210 

0.08610 

0.00040 

0.00060 

2nd year GPA/persist to year 3 

2.0 or higher 

3.0 or higher 

0.14003 

0.06113 

0.00056 

0.00135 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.24036 

0.21407 

0.21331 

0.00039 

0.00046 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 


0.20930 

0.11092 

0.00023 

Bachelor’s degree within 6 years 


0.10643 

— 


Note: Statistically significant (p < .01) variance components are listed. Non-statistically 

significant variance components are noted in the table as ’ and were not included in the final 
models. Variance components for models that could not be developed are left blank. 
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Appendix C 

Differences in Estimated Probabilities of Success for All Students and by 

Developmental Course Grade 

Tables Cl through C 6 


Note: All results pertaining to six-year degree completion were based on only 22 of the 35 four-year 
institutions in the study. 
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Differences between Estimated Probabilities of Longer-Term College Outcomes for Students who Enrolled in Developmental/ 
Standard English Composition and those who Enrolled in only Standard English Composition, for All Students and by Developmental 
English Composition Grade 
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Differences between Estimated Probabilities of Longer-Term College Outcomes for Students who Enrolled in Arithmetic/Elementary 
Algebra and those who Enrolled in only Elementary Algebra, for All Students and by Arithmetic Grade 
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Differences between Estimated Probabilities of Longer-Term College Outcomes for Students who Enrolled in Intermediate 
Algebra/College Algebra and those who Enrolled in only College Algebra, for All Students and by Intermediate Algebra Grade 
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models that could not be developed. Percentiles (Pctl.) correspond to the 5 th and 95 th percentiles of ACT Mathematics scores of students who took the lower-level 
course. Differences for a Passing grade ranged from 0.18 (P5) to 0.1 1 (P95) for a 2.0 or higher GP A/persist to Term 2 (median=0.15). 



Differences between Estimated Probabilities of Longer-Term College Outcomes for Students who Enrolled in Developmental 
Reading/ American History and those who Enrolled in only American History, for All Students and by Developmental Reading Grade 
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models that could not be developed. Percentiles (Pctl.) correspond to the 5 th and 95 th percentiles of ACT English scores of students who took the lower-level 
course. Differences for a Passing grade ranged from -0.02 (P5) to 0.02 (P95) for a 2.0 or higher Year 1 GP A/persist to Year 2 (median=0.00). 
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models that could not be developed. Percentiles (Pctl.) correspond to the 5 th and 95 th percentiles of ACT English scores of students who took the lower-level 
course. Differences for a Passing grade ranged from -0.07 (P5) to -0.02 (P95) for a C or higher grade in Psychology (median=0.03), from -0.07 (P5) to -0.03 
(P95) for a 2.0 or higher Term 1 GP A/persist to Term 2 (median=-0.05), and from 0.17 (P5) to 0.06 (P95) for a 2.0 or higher Year 1 GP A/persist to Year 2 
(median=0.11). 
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Appendix D 

Estimated Probabilities of Success for Full- and Part-Time Students 

Tables D1 through D 6 


Note: All results pertaining to six-year degree completion were based on only 22 of the 35 four-year institutions in 
the study. 
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Table D-l 


Estimated Probabilities of Longer-Term College Outcomes for Students who enrolled in 
Developmental and Standard English Composition, by Full-Time/Part-Time Status 


Outcome variable Estimated probability 

Type Level PT FT 

All students who enrolled in Developmental and Standard English Composition 

^ ^ _ , _ . Cor higher 

Dev. English Comp, grade; first time ^ . 

taken g 

Pass 

0.95 0.96 

0.62 0.66 

0.99 0.99 

Std. English Composition grade ^ 

0.65 0.75 

0.39 0.49 

1st term GP A/persist to term 2 ^ ° r 

3.0 or higher 

0.68 0.84 

0.38 0.47 

1st year GP A/persist to year 2 ° r J/^ er 

3.0 or higher 

0.48 0.61 

0.17 0.23 

2nd year GP A/persist to year 3 ^ ° r 

3.0 or higher 

0.26 0.33 

0.06 0.10 

^ ^ , . . 2.0 or higher 

Cum. GPA at graduation/ ~ , 

, _ 2.5 or higher 

last term _ _ , . t 

3.0 or higher 

0.08 0.19 

0.06 0.16 

0.03 0.08 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 

0.05 0.21 

0.04 0.05 

Bachelor’s degree within 6 years 

0.30 0.46 


Students who enrolled directly in Standard English Composition 


Std. English Composition grade 

C or higher 
B or higher 

0.56 

0.41 

0.86 

0.70 

1 st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.28 

0.18 

0.80 

0.49 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.24 

0.13 

0.63 

0.36 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.13 

0.06 

0.37 

0.19 


2.0 or higher 

0.09 

0.33 

Cum. GPA at graduation/ 
last term 

2.5 or higher 

0.08 

0.31 


3.0 or higher 

0.05 

0.21 

Associate’s degree within 3 years 


0.08 

0.34 

Bachelor’s degree within 5 years 


0.02 

0.15 

Bachelor’s degree within 6 years 


0.13 

0.49 


Note: Shaded cells correspond to full-time/part-time regression coefficients that are not statistically significantly 
different from zero (p > .01). Cells are left blank for models that could not be developed. 
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Table D-2 


Estimated Probabilities of Longer-Term Outcomes for Students who enrolled in Arithmetic and 
Elementary Algebra, by Full-Time/Part-Time Status 


Outcome variable 

Estimated probability 

Type 

Level 

PT 

FT 

All students who enrolled in Arithmetic and Elementary Algebra 

Arithmetic grade; first time taken 

C or higher 
B or higher 
Pass 

0.90 

0.70 

0.96 

0.77 



Elementary Algebra grade 

C or higher 
B or higher 

0.41 

0.29 

0.49 

0.31 

1st term GP A/persist to term 2 

2.0 or higher 

0.61 

0.81 

3.0 or higher 

0.45 

0.52 

1st year GP A/persist to year 2 

2.0 or higher 

0.46 

0.62 

3.0 or higher 

0.23 

0.25 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.19 

0.07 

0.25 

0.12 


2.0 or higher 

0.05 

0.17 

Cum. GPA at graduation/ 
last term 

2.5 or higher 

0.04 

0.15 

ItiiJ V Ivl 111 

3.0 or higher 

0.03 

0.09 

Associate’s degree within 3 years 


0.04 

0.10 

Bachelor’s degree within 5 years 


0.03 

0.09 


Bachelor’s degree within 6 years 


Students who enrolled directly in Elementary Algebra 


Elementary Algebra grade 

C or higher 
B or higher 

0.30 

0.18 

0.47 

0.28 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.34 

0.19 

0.70 

0.34 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.23 

0.09 

0.45 

0.16 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.13 

0.04 

0.25 

0.08 


2.0 or higher 

0.04 

0.12 

Cum. GPA at graduation/ 
last term 

2.5 or higher 

0.03 

0.10 

1 tlu V Ivl 111 

3.0 or higher 

0.02 

0.07 

Associate’s degree within 3 years 


0.03 

0.14 

Bachelor’s degree within 5 years 


0.01 

0.05 

Bachelor’s degree within 6 years 


0.19 

0.42 
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Table D-3 

Estimated Probabilities of Longer-Term Outcomes for Students who enrolled in Elementary and 
Intermediate Algebra, by Full-Time/Part-Time Status 


Outcome variable Estimated probability 

Type Level PT FT 


All students who enrolled in Elementary and Intermediate Algebra 


Elementary Algebra grade; first time g 

taken _ ® 

Pass 

0.89 0.91 

0.60 0.62 

0.93 0.95 

Intermediate Algebra grade S 0r ^[^ er 

00 B or higher 

0.40 0.46 

0.23 0.27 

1st term GP A/persist to term 2 ^ or higher 

3.0 or higher 

0.68 0.82 

0.40 0.47 

1 st year GP A/persist to year 2 2.0 or higher 

3.0 or higher 

0.48 0.59 

0.19 0.24 

2nd year GP A/persist to year 3 ^ or higher 

3.0 or higher 

0.27 0.35 

0.07 0.12 

2.0 or higher 

Cum. GPA at graduation/ _ , , . , 

last term 2.5 or higher 

3.0 or higher 

0.08 0.18 

0.07 0.15 

0.04 0.09 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 

0.05 0.20 

0.03 0.07 

0.39 0.51 

Students who enrolled directly in Intermediate Algebra 

Intermediate Algebra grade S' 0r f 1 ^} 161 

00 B or higher 

0.27 0.57 

0.17 0.36 

1 st term GP A/persist to term 2 2.0 or higher 

3.0 or higher 

0.37 0.79 

0.20 0.42 

1 st year GP A/persist to year 2 2.0 or higher 

3.0 or higher 

0.27 0.58 

0.10 0.26 

2nd year GP A/persist to year 3 ^ or higher 

3.0 or higher 

0.15 0.34 

0.06 0.12 

2.0 or higher 

Cum. GPA at graduation/ _ , , . , 

last term 2.5 or higher 

3.0 or higher 

0.06 0.21 

0.05 0.18 

0.03 0.12 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 

0.07 0.21 

0.02 0.10 

0.12 0.33 
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Table D-4 

Estimated Probabilities of Longer-Term Outcomes for Students who enrolled in Intermediate 
and College Algebra, by Full-Time/Part-Time Status 


Outcome variable 


Estimated probability 

Type 

Level 

PT 

FT 

All students who enrolled in Intermediate Algebra before College Algebra 

Intermediate Algebra grade ; first 
time taken 

C or higher 
B or higher 
Pass 

0.86 

0.56 

0.95 

0.68 

College Algebra grade 

C or higher 
B or higher 

0.51 

0.28 

0.63 

0.37 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.73 

0.45 

0.88 

0.54 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.54 

0.22 

0.69 

0.34 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.32 

0.11 

0.40 

0.15 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.14 

0.12 

0.06 

0.30 

0.26 

0.18 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 


0.14 

0.05 

0.32 

0.13 

Bachelor’s degree within 6 years 


0.42 

0.53 

Students who enrolled directly in College Algebra 

College Algebra grade 

C or higher 
B or higher 

0.42 

0.27 

0.73 

0.52 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.21 

0.13 

0.81 

0.51 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.19 

0.09 

0.64 

0.39 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.10 

0.05 

0.37 

0.21 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.06 

0.06 

0.04 

0.34 

0.31 

0.24 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.06 

0.02 

0.10 

0.32 

0.18 

0.53 
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Table D-5 

Estimated Probabilities of Longer-Term Outcomes for Students who enrolled in Developmental 
Reading and American History, by Full-Time/Part-Time Status 


Outcome variable 


Estimated probability 

Type 

Level 

PT 

FT 

All students who enrolled in Developmental Reading before American History 

Developmental Reading grade ; first 
time taken 

C or higher 
B or higher 
Pass 

0.81 

0.48 

0.98 

0.93 

0.72 

0.99 

American History grade 

C or higher 
B or higher 

0.43 

0.20 

0.60 

0.33 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.61 

0.35 

0.83 

0.46 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.40 

0.11 

0.60 

0.23 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.20 

0.04 

0.36 

0.11 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.07 

0.05 

0.02 

0.19 

0.15 

0.08 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.05 

0.07 

0.13 

0.20 

0.16 

0.28 

Students who enrolled directly in American History 

American History grade 

C or higher 
B or higher 

0.54 

0.34 

0.77 

0.55 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.32 

0.20 

0.78 

0.49 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.27 

0.14 

0.61 

0.37 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.16 

0.07 

0.38 

0.20 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.10 

0.09 

0.06 

0.33 

0.30 

0.23 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.09 

0.04 

0.17 

0.33 

0.19 

0.50 
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Table D-6 

Estimated Probabilities of Longer-Term Outcomes for Students who enrolled in Developmental 
Reading and Psychology, by Full-Time/Part-Time Status 


Outcome variable 


Estimated probability 

Type 

Level 

PT 

FT 

All students who enrolled in Developmental Reading before Psychology 

Developmental Reading grade ; first 
time taken 

C or higher 
B or higher 
Pass 

0.80 

0.54 

0.98 

0.94 

0.76 

0.98 

Psychology grade 

C or higher 
B or higher 

0.49 

0.25 

0.69 

0.41 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.59 

0.37 

0.83 

0.46 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.39 

0.11 

0.59 

0.24 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.22 

0.05 

0.34 

0.11 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.07 

0.05 

0.02 

0.18 

0.15 

0.09 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 


0.05 

0.04 

0.19 

0.07 

Bachelor’s degree within 6 years 


0.20 

0.37 

Students who enrolled directly in Psychology 

Psychology grade 

C or higher 
B or higher 

0.62 

0.43 

0.83 

0.63 

1st term GP A/persist to term 2 

2.0 or higher 

3.0 or higher 

0.34 

0.22 

0.79 

0.50 

1st year GP A/persist to year 2 

2.0 or higher 

3.0 or higher 

0.28 

0.14 

0.61 

0.36 

2nd year GP A/persist to year 3 

2.0 or higher 

3.0 or higher 

0.15 

0.07 

0.36 

0.20 

Cum. GPA at graduation/ 
last term 

2.0 or higher 
2.5 or higher 

3.0 or higher 

0.09 

0.08 

0.06 

0.31 

0.29 

0.22 

Associate’s degree within 3 years 
Bachelor’s degree within 5 years 
Bachelor’s degree within 6 years 


0.09 

0.04 

0.18 

0.30 

0.18 

0.53 
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